Background: In Nigeria, there is a plethora of evaluators found in the over 90 universities, specialised educational institutions and private research organisations. However, there is limited or no opportunity for networking among the evaluators from similar and different programmatic specialisations. After applying the determinant framework to assess the evaluation capacity development situation in Nigeria, we agree on the importance of supporting the establishment of a National Voluntary Organisation for Professional Evaluation (VOPE).
Objectives: Several leaders in evaluation were competing recently for occupying the national space reserved for a National VOPE. The main objective was to encourage partnership.
Methods: We used a powerful theoretical framework provided by game theory and new institutional economics. We analysed the situation and identified the challenge they are facing as a Nash Equilibrium-of-a-Game View of Institutions: each player knows the equilibrium strategies of the other players and no player has anything to gain by changing only his or her own strategy. This explains why Nigerian evaluation leaders were not able to cooperate for the last two decades.
Results and conclusion: To break this barrier, we proposed a new deal to the leaders that had the advantages of reshaping the ‘rules of the game’. We proposed a federation of associations, akin to a coalition in game theory. The result was that all leaders came together under this umbrella organisation, to celebrate the evaluation year in 2015 and committed under the Abuja Declaration on Evaluation to register and establish an association, with an elected board, a written constitution and election bylaws. The association is governed by a Board of Trustees, which is chaired by the former Minister of Planning. Elections are planned for the end of 2017.
Capacity development is about transformations that empower individuals, leaders, organisations and societies (UNDP 2009). The process involves multiple and complex interlinked factors that contribute to each other. Capacity is defined as ‘the ability of people and organisations to define and achieve their objectives’ (OECD DAC 2006). So what is evaluation capacity development (ECD) and how is it defined?
Evaluation capacity development is understood as:
the process of unleashing, strengthening and maintaining evaluation capacities. Capacity development is a long-term, endogenous change process that takes place in the context of ongoing partner and donor efforts to strengthen related systems of management, governance, accountability and learning, to improve development effectiveness. The best capacity development approaches are flexible, adapted and sustainable. (World Bank 1998:5)
Capacity development involves three interdependent levels: individual, organisational and the enabling environment. These interrelated capacities function together and reinforce one another – all three are necessary for a functioning system.
Why is evaluation capacity development important?
According to the World Bank:
The development of national or sectoral evaluation systems – known as Evaluation Capacity Development (ECD) – is increasingly recognized as an important aid to sound governance and as a means to help achieve high levels of public sector performance. ECD ensures that evaluation findings are available to assist countries in four key areas. (Mackay 1999:1–2)
The four key areas are the following:
- budget allocation
- understanding and improving ongoing performance
- accountability purpose and reporting back on the use of public monies
- demonstrating programme effectiveness and marketing successes.
Evaluation capacity development is at the heart of good governance, and evaluation contributes to at least three out of the six key dimensions of good governance (Jobin & Adrien 2008:102–118). Indeed, Kaufmann (1999) identifies six dimensions of good governance within which the evaluation function directly contributes to three of these dimensions:
Voice and Accountability: Evaluation provides opportunities to right holders to voice their concerns and appreciation of services provided by duty bearers and by doing so enhances trust in and transparency of public institutions, which in turn reduces transaction costs. It also reduces information asymmetry between the Principal (Government) and Agents (Citizen) on government performance, hence strengthening the accountability function.
Government Effectiveness: Evaluation provides opportunities to duty bearers to improve performance by learning from experience and understanding what works where, why and how. It also helps to ensure performance is achieved, by being efficient, effective and ensuring value for money. By better understanding how performance is achieved and how governance links up to it, evaluation helps organisations to minimise transaction costs (Williamson 1985).
Corruption: Evaluation deters corruption, as projects and programmes under scrutiny are more likely to reveal any corruption than those that are not, thus improving performance and reducing transaction costs.
In a context where greater accountability because of the diminishing public resources available is required, and where public funds and public aid are channelled through development partners with greater scrutiny and there are international agreements such as the Paris Declaration and the Busan Partnership for Effective Development Cooperation to increasingly use national systems, development of evaluation capacities is relevant more than ever.
Why is supporting the establishment of a Voluntary Organisation for Professional Evaluation (VOPE) so important? Segone (2010) developed a conceptual framework to analyse the support that is needed at different capacity levels (individual, institutional and external enabling environment). In order to successfully develop capacities, we believe that supporting the enabling environment is first among equals, given that it supports development of other levels of capacities.
It is also increasingly recognised that there is a link between the VOPE and ECD, which can be unpacked from both the supply side and the demand side (Segone & Rugh 2013). Indeed, from the supply side, it is expected that the VOPE will play a greater role in professionalising the in-country evaluation services and develop, for instance, national standards, engage its members in capacity-building activities, encourage innovation and use of new methods and approaches and strengthen good practices that fit local custom and so on. The VOPE will naturally over time seek to strengthen its control of the access of the quasi-rent, to modulate the practices and evaluation profession in Nigeria and to control quality, thus establishing trust between them and the public.
From the demand side, the role of the VOPE is not any less important. The VOPE’s primary role is to ensure that government policy exists for a strong and effective evaluation function and to advocate for it. For instance, VOPEs create spaces for debates by promoting discussions within government-relevant circles, such as engaging parliamentarians, and public media in the importance of their affairs. They can also work and partner with universities and vocational institutions to ensure that educational programmes exist and universities deliver diplomas that enrich and are recognised by the evaluation profession. There is an increase in the number of publications on the importance of, and the roles of, the VOPEs, in the enabling environment and in ECD. The links are strong (Rist, Martin & Boily 2011:89).
The case study presented in this paper refers to support of the enabling environment for ECD in Nigeria by strengthening simultaneously both the supply and the demand sides for evaluation. It describes the journey taken by Nigerian evaluation leaders, the Ministry of Budget and National Planning (MBNP) and development partners to support the establishment of a VOPE in Nigeria.
This paper proposes an innovative approach that informed a strategy aimed at supporting ECD in Nigeria. We used a robust theoretical framework derived from the new institutional economics (NIE) and game theory, popularised among others by Douglas North and John Nash, both Nobel Laureates. The application of the theoretical framework generated concrete prescriptive actions that informed and guided the authors on how to proceed to operationalise the strategy and successfully address the problems that they were facing.
The paper is structured as follows: first, the descriptive context and the ‘problem’ faced by the Nigerian government and the supporting of development partners in establishing an effective VOPE are discussed. Then the key concepts used to inform a strategy in designing an effective intervention to support the MBNP in addressing the problem and reinterpreting it in light of the adopted theoretical framework are presented. Finally, the results and a way forward in fulfilling the establishment of a sustainable VOPE in Nigeria conclude the paper.
Evaluation capacity development: Barriers and challenges in Nigeria
Nigeria with a population of over 186 million people (CIA 2017) is the most populous country in Africa and has the fastest growing population on the continent. In terms of governance, Nigeria is composed of 774 local governments, 36 states plus federal capital territory and a federal government. Each tier of government in Nigeria implements policies to fulfil the rights of its citizens, especially those of women and children. Accountability and evidence for and about the attainment of human rights relating to health, education, protection, water and sanitation are key functions of a national evaluation system and culture. With the drive to achieve the Sustainable Development Goals (SDGs), Nigeria has been the host of many development initiatives from non-governmental, governmental and intergovernmental organisations every year. All these programmes, projects and initiatives need to be evaluated to ensure that the implementation meets the goals and targets and to draw lessons for future plans.
In Nigeria, there is a plethora of evaluators found in the over 90 universities and other specialised educational institutions as well as private research organisations. However, there is limited or no opportunity for networking among the evaluators from similar and different programmatic specialisations. There is also a need to maintain and consolidate, develop and upgrade the skills and competencies of Nigerian evaluators.
Nigerian evaluators currently work in isolation, with little support from peers. The implication is that evaluators in Nigeria are rarely trained in well-established evaluation approaches, methodologies and standards that will make them compete favourably with international evaluators, thereby supporting a country-led evaluation system.
Furthermore, Nigerian evaluators lack the capacity to mobilise resources to facilitate effective networking and sharing of knowledge within and between zonal and programmatic concerns. There are few attempts to nurture advanced-level evaluation expertise, to promote training placed in Nigerian contexts and evaluation approaches or to highlight Nigerian evaluation expertise on continental and global platforms. Thus, demand for a country-led evaluation is low and the use of evaluation for learning and decision-making is limited and dominated by the demands for accountability to international donors. This situation undermines the needs for greater ownership and accountability by development partners.
Back in 2014, the authors analysed the specific barriers and bottlenecks facing ECD in Nigeria. The determinant framework was developed by United Nations Children’s Fund (UNICEF) to help identify the key determinants affecting a given intervention, and more particularly to understand the barriers, bottlenecks and enabling factors that either constrain or advance the achievement of desired outcomes for disadvantaged children.
It emphasises strengthening the capacity of government and partners to regularly monitor interim results to enable more effective programme implementation and timely course corrections in plans and strategies at all levels. (UNICEF 2013)
The application of the determinant framework was also informed by a thorough analysis of the ‘readiness assessment’ of development partners for an effective evaluation function. (Kusek & Rist 2004)
The results of the analysis of applying the determinants framework are presented in Table 1a and Table 1b.
|TABLE 1a: Determinants of the evaluation capacity development in Nigeria.
|TABLE 1b: Determinants of the evaluation capacity development in Nigeria.
From the analysis of determinants performed, it was clear that one key element of support needed and agreed upon with government was support for the establishment of a National VOPE (Segone & Rugh 2013). This support was strategic as it simultaneously helped to address several barriers. Indeed, over the last few decades, the number of VOPEs (national and regional VOPEs) has risen from 15 in the 1990s to more than 155 by early 2013. The aggregate total of their memberships now surpasses 34 000 (Segone 2014:6). This number keeps increasing.
Since 2004, several evaluation networks have existed. Indeed, Nigerian evaluation leaders have participated in the African Evaluation Association (AfrEA) Conferences held in Niamey and Cairo in 2007 and 2009. After each conference, participating Nigerians felt the need for and were challenged to try to establish a National Association of Evaluators in Nigeria. For instance, after the Cairo Conference (2009), the Monitoring and Evaluation Network of Nigeria was put together. However, after the Niamey Conference, another group coming out of the conference proposed an evaluation society and network, which gravitated around the MDG offices. All such networks are registered by AfrEA and the International Organization for Cooperation in Evaluation (IOCE).
However, none of these two bodies has the will or the authority to recognise the ‘legitimacy’ or ‘appropriateness’ of their VOPE membership. In other words, what does one need to become a recognised VOPE? What are the key ingredients? This point is important because it creates a vacuum that generates incentives, which encourages different players to exploit to their advantage at the country level; they compete to get control of and access to the monitoring and evaluation (M&E) quasi-rent in Nigeria. It is also important for these organisations to define and clarify through their membership base what it takes to become a member and what the core elements or ingredients are that make a VOPE a VOPE.1
Other evaluation stakeholders and leaders and VOPEs in Nigeria have also emerged since and both have achieved some significant results in their respective functions such as membership mobilisation, participation and recognition of Nigeria in the global evaluation map, as well as catalysing attention and discussions on evaluation in Nigeria. However, the non-cooperative environment in which they were locked up because of their dominant equilibrium strategy2 resulted in poor sharing of information and ideas, low capacity development and slow institutionalisation of evaluation in the three tiers of government. This proliferation of networks claiming to be the primary legitimate National VOPE was not the fruit of chance, but was rather stimulated by strong incentives and dominant strategies that we need to describe in the next section.
Method and approach: Understanding the ‘rules of the game’ and related structure of incentives
Since the end of 2010, informal discussions were held with active VOPEs in Nigeria with the overall objective of enhancing the professionalisation of Nigerian evaluators, through the means of a national, united and strong Nigerian Evaluation Association. The next section of this paper will present the theoretical framework used by the authors to inform the design of a strategy that addresses the specific challenges in supporting the establishment of a VOPE in Nigeria because of the non-cooperative environment in which existing leaders (players) were operating.
To address the problem of non-cooperative environment among key stakeholders and national partners, the authors needed to understand the root causes of the problem and answer some basic but very important questions: what is the structure of incentives shaping the evaluation leaders interactions (players), and what are their motives and respective strategies?
To inform the strategy, the authors used two pertinent theories that were combined in solving social dilemma and economic problems associated with collective action. Let us succinctly present the theories here and their key concepts.
The new institutional economics
Evaluation capacity development and the application of evaluation both take place in institutional settings influenced by financial and non financial incentives. The NIE provides a useful and robust framework for analysing such situations. The NIE is an interdisciplinary enterprise combining economics, law, organisation theory, political science, sociology and anthropology in order to understand the institutions of social, political and commercial life. It is a branch of economics that deals essentially with institutions, explaining institutional change and associated costs. The NIE has been influenced by many authors, many of whom are Nobel Laureates who studied the different problems of the collective action, such as moral hazard, adverse selection and bounded rationality (The New Institutional Economics 2017). For this case study, the authors are using the definition of institutional arrangements as follows:
Institutions are the rules of the game in a society; more formally, they are the humanly devised constraints that shape human interaction. In consequence they structure incentives in exchange, whether political, social, or economic. Institutional change shapes the way societies evolve through time and hence is the key to understanding historical change. (North 1990:3–5)
According to North (1990), the key features of institutions and institutional arrangement are the following:
- Institutions reduce uncertainty by providing structure to everyday life.
- Institutions include any form of constraint that human beings devise to shape human interaction. They can be either formal, such as rules that human beings devise, or informal constraints such as conventions and codes of behaviour.
- Institutional constraints include both what individuals are prohibited from doing and, sometimes, under what conditions some individuals are permitted to undertake certain activities.
- And More importantly, they represent the rules of the game and the structure of incentives that shape human interactions one way or another.
The NIE differs from the old institutional economics as it rejects a key assumption of access to perfect information. The NIE assumes that in the presence of uncertainty, information problems and ‘transaction costs’, economic agents cannot gather all the information necessary to calculate, case by case, the optimal solution for the problems they face. Individuals are therefore ‘intendedly rational but only limitedly so’ (Williamson 1985:45). They thus need to develop strategies to achieve their economic goals.
In other words, institutions matter (North 1990).
Game theory and Nash equilibrium
As noted above, North (1990) identifies institutions with formal rules such as constitutions, statutory laws and contracts, as well as informal rules such as social norms, beliefs and custom. Those rules may be considered in a game form, representing the specific parameters of what is ‘permissible’ and what is not and the constraints on the sets of players’ action choices.
Von Neumann game theory has been used in mathematical studies, economics, social science and policy studies. Von Neumann game theory seeks to understand individual behaviour ‘based on a careful analysis of the ordinary every day interpretation of economic facts’ (Von Neumann & Morgenstern 1944:7). The most well-known problem in game theory is probably the Prisoner Dilemma, where two individuals decide not to cooperate despite it being in their interest to do so. This example is used to understand non-cooperative games and their impact on the collective good.
There are different types of games, and some can be cooperative if the players are able to form binding commitments externally enforced (e.g. through contract law). Cooperative or non-cooperative games (prisoner dilemma): a game is cooperative if the players are able to form binding commitments. For instance, the legal system requires them to adhere to their promises.
A game is non-cooperative if players cannot form alliances or if all agreements need to be self-enforcing (e.g. through credible threats). A game is considered to be non-cooperative when players are not able or do not have the incentive to bind in commitment and are pursuing self-interest utility.
Alternate-move games (like chess): players take turns, strategies involve many moves and can evolve during play.
Simultaneous-move games (like rock–paper–scissors): players commit to a move without knowledge of the other players’ strategy.
Zero-sum games (games of ‘total conflict’): one player’s gain equals the other player’s loss. If we sum the payoffs at each outcome, the result is always zero. Play against nature: deciding to take the umbrella or not when leaving for work in the morning is often used as example for this type of game. The Normandy invasion in 1942 was also studied to understand the decision that led to D-day given the hazard associated with the weather (Macrae 2010). We assume players are rational in the sense that they seek the outcome where the resulting payoff is in their best interest.
There are also constant-sum games, in which choices by players can neither increase nor decrease the available resources. In zero-sum games, the total benefit to all players in the game, for every combination of strategies, always adds to zero (if one wins, the other loses the equivalent; hence, zero-sum).
Lastly, another type of game that is relevant to the case presented here is the coalition game. Coalition game studies the competition between groups of players who may have interest in cooperation in order to achieve a greater benefit, the core. The core is the set of payoff that is generated by the coalition and provides the structures of incentives to remain in the coalition, a benefit greater than one of defecting and leaving the coalition.
Combining new institutional economics and game theory: A Nash Equilibrium-of-a-Game View of Institutions
Game theory provides a conceptual framework for the analysis of self-enforcing institutions. It enables us to deductively restrict the set of (rational) shared beliefs capturing individuals’ expectations with respect to actions that others will take in various contingencies and circumstances. Participants in a game are called players or agents. The games studied can be quite serious and are studied in many areas of the natural and social sciences and, of course, economics and politics. Players may have one or more possible options for play. These options are referred to as strategies: dominant or dominated strategies. We study the rational selection of strategies, the interaction of players and the resulting outcomes.
Schotter (1981) introduced the idea of the ‘institution as the equilibrium outcome of a game’. For him, an institution is not described by the (exogenous) rules of a game, as representatives of the NIE like North (1990) do, but as a (Nash-) equilibrium of a repeatedly played non-cooperative game, that is, a state in which no agent (player) has an incentive to deviate from his action plan provided no other agent deviates. In other words, the equilibrium outcome of a game becomes the institution that shapes human interaction.
Nash equilibrium may be better understood as a solution concept of a non-cooperative game involving several players in which each player is assumed to know the equilibrium strategies of the other players, and no player has anything to gain by changing only their own strategy. Indeed Osborne and Rubinstein (1994) described Nash equilibrium as follows: ‘the most commonly used solution concept is that of Nash equilibrium. This notion captures a steady state of the play of a strategic game in which each players holds the correct expectation about players’ behaviour and act rationally’ (Osborne & Rubinstein 1994, p. 14).
So for Schotter (1981) and Furubotn and Richter (2005), a Nash equilibrium can be seen as a self-enforcing agreement, that is, an (implicit or explicit) agreement that, once reached by the agents, does not need any external means of enforcement because it is in the self-interest of each player to follow the agreement if the others do. Indeed, for them, an institution is defined as a salient Nash equilibrium of a recurrent ‘supergame’ (Wikipedia 2017) about the way a given ‘underlying game’ (e.g. a prisoner’s dilemma game) is repeatedly played (see Box 1). It is self-enforcing in the sense that it is in each actor’s own interest not to deviate from their decision (‘not to stray from the straight and narrow’). The term ‘force’ is misleading since no brachial ‘force’ is applied only the ‘force’ of each actor’s own interest (Richter 2008: 19).
|BOX 1: Prisoner Dilmma: A Nash equilibrium sample.
Discussion: Applying the approach to the strategy in Nigeria
It was important to introduce these key concepts because they inform the analysis of the situation and the strategy that was designed to address the problems identified. In view of the above, in 2013 the authors agreed on the need and importance of establishing a National VOPE in Nigeria, which would be able to effectively identify some of the barriers and bottlenecks in the application of the Determinant Framework Analysis. Thereafter, the MBNP formally requested collaboration with UNICEF on facilitating the mobilisation of practitioners towards establishing an association (VOPE) that will serve as a rallying point for evaluators and a point of convergence for all the major diverse evaluation platforms and evaluation enthusiasts in the country.
Several evaluation leaders and stakeholders have existed in Nigeria for the last decade or so, as described above, all of them claiming to be the only national legitimate VOPE. There are a few powerful incentives that explain this situation and which are associated with the monitoring of the MDGs projects, (MDGs projects and/or any other governmental or donors projects) and thus the accessing of MDG funds to do so, or a quasi-rent. A quasi-rent was introduced originally by Tullock (1967) as a result of rent-seeking behaviour. The idea is that private individuals will seek to create an artificial monopoly to gain wealth, without creating any. As a result, it increases income inequality and poor allocation of resources by creating an advantage to one competitor through the imposing of restrictions on the other competitors. Other incentives were related, such as getting access to the other quasi-rents from the donor-led monitoring and evaluation opportunities and contracts. In addition, being represented on the board of the National Association and having enough visibility to benefit from international capacity-building projects such as EvalPartners and donor-led ECD initiatives and having opportunity to get invited or co-opted in different international meetings and events under the umbrella of capacity development provided enough incentives for resisting collaboration with other national leaders. A last, but not least, incentive is derived from being on the board of a VOPE, which gives the appearance of adequate competence and skills for conducting evaluation studies and strengthens credibility and national ownership – qualities sought by donors. Hence, it increases the attractiveness of the leaders for accessing the quasi-rent.
Thus, claiming to be the primary and only national VOPE provides an excellent entry point for accessing national and international quasi-rent and engaging with governments and donors, and thus accessing the quasi-rent of M&E opportunities, while excluding others. There is nothing new in this analysis. It is a fundamental role of any professional association to control and access the quasi-rent: the right to control the access of practising a profession. This analysis has also been done in auditing, especially Arrunada (1999) who examined how auditors manage their profession and the approach they have embraced for managing the quality of an audit. The key message here is that it is important to understand the structures of incentives that surround an intervention.
While we recognise that this is an oversimplification, the key messages here are that there are real and tangible incentives and benefits in establishing a National VOPE. This may explain in part the proliferation of several VOPEs in one country and which are perhaps not necessarily a consequence of being member-driven (different views and needs in the membership base resulting in a need for several associations). These incentives are generated by national institutional arrangements as they relate to budget and M&E contracting opportunities within a given country. These incentives are also generated by international initiatives, such as those promoting ECD and one key condition to access these opportunities within a given country, being a board member of a National VOPE, has its advantages. Consequently, for Nigerian stakeholders there were neither incentives to share the benefits in accessing the quasi-rent nor institutional arrangements in place that encouraged cooperation among national leaders.
Another important rule of the game, which explains the non-cooperative environment that Nigeria found itself trapped in, is that there should only be one National VOPE in the country. Competition and coexistence was simply not recognised or allowed because it was perceived as sharing the benefits of the M&E pie or quasi-rent. As a consequence, the dominant strategy of the players was ensuring that they are the only recognised national association, thus undermining the other leaders’ efforts in positioning themselves as national leaders, hence the zero-sum game mode. Indeed, during the AfrEA Conferences in 2006 and 2009 and in other international meetings, key Nigerian leading networks were sending representatives to represent Nigeria in the international fora, and also pursuing the strategy of and claiming the monopoly of the space they occupy in the motherland. The dominant strategy leading to Nash equilibrium of non-cooperation was closely linked with this informal rule. This rule was confirmed by efforts made by leading networks to government authorities (Minister of National Planning, Permanent Secretary, MDGs Office and senior advisor to President, etc.). Nigeria was trapped in an institutional arrangement of the Nash equilibrium type: each player knows the equilibrium strategies of the other players, and no player has anything to gain by changing only their own strategy. Nigeria has been trapped in this situation for the last two decades.
These incentives were strong enough to encourage various leaders to compete for accessing the benefits of leading the National VOPE, thus creating a non-cooperative environment in Nigeria for the last few decades. Several attempts to address this Nash equilibrium barrier were made but all failed, until now.
From a game theory point of view, the rules of the game could be summarised as follows:
- It is in the nation’s best interest to have only one VOPE.
- As a consequence, the dominant strategy for each player was not to cooperate and thus exclude others.
- The main benefit was to control the access of a quasi-rent within the country and with development partners.
- The dominant strategy was to claim the monopoly of the space of a National VOPE:
- To undermine other leading network efforts to move ahead by any possible means.
- To organise local events to give more credibility to the claim in the first point above.
- To advocate within the political circles (national and international) for a space that is monopolistic in nature.
Informed from this understanding of the rules of the game and knowing the key players’ dominant strategies and what was at stake, the strategy for the authors was thus simple: to find a way to change the rules of the game.
Dominant strategy: Non-cooperative game – No deal
To participate and self-sponsor themselves to the relevant events (Eval Partners, AfrEA Conferences or others subscribing to IOCE database) to make their respective claims about being the only ‘genuine’ National VOPE in Nigeria in a monopolistic manner.
Dominated strategy: Collaborating with the others
The VOPE to coordinate for the access and control of the quasi-rent as provided by development partners (in a context where national resources for evaluation are limited or non-existent) in a coalition.
In summary, if one VOPE cooperates while the others do not cooperating, the one cooperating is losing all, while the one not cooperating is winning all (zero-sum game), getting the national recognition. Hence, the dominated strategy in all scenarios is to not cooperate thus reaching the Nash equilibrium as an institution. The gain in cooperating is greater than in not cooperating, but given the lack of trust and competitive nature, each player pursues the strategy of not cooperating, which provides more gain to them, if others cooperate (or abdicate). This non-cooperative game is presented in Figure 1.
FIGURE 1: Non-cooperative game for Voluntary Organisations for Professional Evaluation (VOPEs) in Nigeria.|
The vision: Federation of Associations of Evaluators in Nigeria
The authors proposed an intervention, which was designed to change the rules of the game and propose stronger incentives to cooperate. Leaders accepted the new deals and agreed to work together, a first in two decades. We therefore moved away from a non-cooperative game to a cooperative game proposing a coalition game type. To succeed the benefits (referred to as the Core, Wikipedia 2017) of cooperation must be perceived as more significant than the benefits of defection (see Figure 2).
FIGURE 2: A new deal moving the Nash equilibrium from a coalition game institutional arrangement.|
With the proposed vision, the authors actually offered a new deal3: to create a federation of associations. The proposed vision came alongside principles that were introduced by the government (or new rule of the game), to create the new rules of the game:
- Inclusiveness and encouraging cooperation: There will be room for everyone who has an interest in supporting the professionalisation of evaluation.
- Majority-based decision: The need to agree to disagree and decisions based on the views of the majority.
- The right to coexist: Every stakeholder has a right to exist. This was set up to encourage cooperation by diminishing the importance of defecting.
- The right to exit: Should one stakeholder wishes to exit, then they can and they can come back at any point in time. This principle provides a safeguard against defection. By explicitly acknowledging defection, it was more difficult to use the threat of defection as a negotiation position and use the hold-up problem.4
The project was led by MBNP. Because of the wider partnership between government and development partners (among 15 international development partners), the quasi-rent generated by this vision significantly gained importance in view of all the evaluation leaders (players). In 2014 and 2015, the authors held stakeholder meetings in Kaduna, Lagos and Abuja, meeting more than 150 leaders and stakeholders in the country. They communicated the vision and then listened to the feedback provided by the stakeholders during the plenaries. It was also an opportunity to inform the players about the new rules of the game and the changing incoming institutional landscape.
This vision offered a new institutional arrangement5 with the incentives to move away from a non-cooperative zero-sum game towards a new Nash equilibrium of a coalition game.6 Many players had strong incentives to collaborate, given the size of the anticipated benefits (Core) where a new Nash equilibrium is generated to instead encourage cooperation: the benefit of cooperating needs to be perceived as greater than the benefits of not cooperating. The proposed vision also contained elements that the authors believed were necessary for an evaluation network to be considered an evaluation association (VOPE). They were important to articulate and set up front (as game rules changes), so evaluation leaders in Nigeria raised the bar at entry level and made it more difficult for opportunistic behaviour to adhere. It is more difficult now to claim that one is the National Evaluation Association, if these elements are considered the minimum for being a VOPE. The new rules were the following:
- Having a written constitution that governs the affairs of the association and which is endorsed by the member base
- Having an elected board, which is democratically elected from the membership base
- Having written bylaws, endorsed by the members
- Presenting the status of the conduct of the affairs of the Association during an Annual General Meeting (annually or bi-annually).
By setting these straightforward principles, or virtues, the authors also offered an institutional arrangement (a deal) that was more structured, transparent and unbiased and offered an equal chance to all to participate (equitably). The authors also established a Board of Trustees (BOT) that is chaired by the Minister of Budget and National Planning, and which plays a key role of enabler and watchdog.
Conclusion: Towards a federated Nigerian Voluntary Organisation for Professional Evaluation in 2017–18
Following the nationwide consultations, in 2015 all stakeholders that were competing for the National VOPE space agreed to collaborate and to work together in the evaluation year 2015. They worked towards the celebration of the evaluation year 2015 with a conference with the theme ‘Evaluation and good governance: Using evaluation to improve lives through better policymaking’.
During this conference, all stakeholders present unanimously agreed to a common vision and signed what is now recognised as a precious milestone: the Abuja Declaration on Evaluation.7
The Abuja Declaration set the parameters to establish a vibrant evaluation association. It is intended that this association will develop and promote an appropriate level of professionalism and collaboration with all stakeholders to build a community of evaluators and practitioners for the promotion and institutionalisation of evaluation in Nigeria. It also intends to provide services for its members, promote the use of evidence-based decision-making processes and advocate support, and constructively demand the use of evaluation by Nigerian governments, so as to bridge the gap between demand and supply of evaluation in Nigeria.
The Abuja Declaration on Evaluation also established the commitment of all evaluation leaders supporting the intervention:
- We recognise the importance of country-led monitoring and evaluation systems for the effective development of Nigeria.
- We recognise that monitoring and evaluation systems at all levels – federal, states and Local Government Areas (LGAs) – are paramount in ensuring the achievement of SDGs as endorsed by the government of Nigeria.
- We understand that the SDGs and related targets have the potential to transform Nigerian society. We also understand that the efforts that will lead to SDGs need to be inclusive, transparent and embedded with the virtues of good governance and being accountable.
- We, the evaluation practitioners in Nigeria, are entrusted by the highest regard for the respect of those virtues and shall protect, promote and defend them.
- We recognise the important role of evaluation in good governance.
- We commend the establishment of the Nigerian Association of Evaluators (NAEs).
The Abuja Declaration on Evaluation also created an Interim Executive Committee (IEC) to ensure that the Nigerian VOPE is fully established and agreed to:
- Register the NAE under Nigerian law.
- Finalise and propose for full membership adoption of the constitution and election by-laws that will govern the NAE.
- Conduct a fair and transparent election to select an executive board that will manage the NAE under the oversight of the trustees.
- The IEC members agreed not to stand for election for the next executive committee.
- Agreed that whoever wants to stand for election had until 31 December 2015 to step down and state their intentions.
These commitments were set to establish the rules of the game by which the leaders agreed to abide. They were useful as they were made public and written down. In order to ensure that the interim committee met its own commitments, the stakeholders established a BOT chaired by the former Minister of National Planning and recognised evaluation leaders in Nigeria. The BOT served as an institutional safeguard mechanism that ensured that the commitments taken by the evaluation leaders of the IEC were implemented, as well as a supporting mechanism to ensure that the IEC received whatever support it needed.
Once fully established, the association will advocate for and monitor the increase in awareness of the need for country-led evaluation studies by the executive and legislative arms of government at national, states and LGA levels. It will be monitored by the Association as evidenced by the increased number of related statements by policymakers and increased budgetary allocation to M&E budgetary items in the annual National Budget.
The MBNP also took several commitments in 2015, among which was the adoption of a National Evaluation Policy that will govern the evaluation function at a federal level. This has the potential to generate country-led evaluation studies and strengthen the supply side.
The BOT met in November 2016 with the representatives of the interim committee to get an update from the interim committee. The VOPE is ready to be registered under Nigerian law. There is an agreed upon constitution ready to be endorsed by the membership base. Furthermore, it was agreed to support the interim committee, by allowing the BOT to conduct and provide the oversight for the election of an elected board, in February 2017. The transitional arrangement will last until a conference bringing the members together can endorse the constitution under a full Annual General Meeting by the end of 2017 under the lead and support of actual Minister of State for MBNP.
The authors would like to thank Mr Mahesh Patel for his thoughtful comments on earlier versions. Of course, if any mistakes occur, they are the authors’ responsibility only.
The authors declare that they have no financial or personal relationships that may have inappropriately influenced them in writing this article.
Theoretical approach and methodology developed by D.J. The context and data collection through the consultation were facilitated by Z.L. The write-up was drafted by D.J. and commented on and contributed to by Z.L.
Arrunada, B., 1999, The economics of audit quality: Private incentives and the regulation of audit and non-audit services, Springer, New York.
CIA, 2017, The world fact book, viewed n.d., from https://www.cia.gov/library/publications/the-world-factbook/geos/ni.html
Furubotn, E.G. & Richter, R., 2005, Institutions and economic theory. The contribution of the new institutional economics, 2nd edn., University of Michigan Press, Ann Arbor.
Jobin, D., 2014, Application of the UNICEF determinant framework analysis to evaluation capacity development in Nigeria, UNICEF Rolling Work Plan with MBNP 2014–15, UNICEF, New York.
Jobin, D. & Adrien, M.-H., 2008, Country-led monitoring and evaluation systems: Better evidence, better policies, and better development results, UNICEF, viewed 06 February 2017, from http://mics.unicef.org/files?job=W1siZiIsIjIwMTUvMDEvMzAvMDMvMjUvNTUvNTAwL0NvdW50cnlfbGVkX01FX3N5c3RlbXMucGRmIl1d&sha=cdcc92ec34d8448b
Kaufmann, D., Kraay, A. & Zoido-Lobatón, P., 1999, Governance Matters, Policy Research Working Paper No. 2196, World Bank, Washington, DC.
Kusek, J.Z. & Rist, R.C., 2004, Ten steps to a results-based monitoring and evaluation system, World Bank, Washington, DC.
Mackay, K., 1999, Evaluation capacity development: A diagnostic guide and action framework, ECD Working Paper No. 6, World Bank, Washington, DC.
Macrae, N., 2010, John Von Neumann: The scientific genius who pioneered the modern computer, game theory, nuclear deterrence and much more, AMS, p. 304, Providence, RI.
North, D.C., 1990, Institutions, Institutional Change, and Economic Performance, Cambridge University Press, Cambridge.
OECD DAC, 2006, The challenge of capacity development: Working towards good practice, OECD, Paris.
Osborne, M.J. & Rubinstein, A., 1994, A course in game theory, MIT, Cambridge, MA.
Richter, R., 2007, ‘The market as an organization’, Journal of Institutional and Theoretical Economics 163(3), 483–492.
Richter, R., 2008, On the New Institutional Economics of Markets, paper prepared as basis of a lecture held at the Faculty of Sociology, Bielefeld University, Bielefeld, Germany, viewed 10 November 2016, from http://www.uni-saarland.de/fak1/fr12/richter/institut/nie_of_markets.pdf
Rist, R., Martin, F. & Boily, M.-H., 2011, Influencing change: Building evaluation capacity to strengthen governance, World Bank, Washington, DC.
Schotter, A., 1981, The economic theory of social institutions, Cambridge University Press, New York.
Segone, M. (ed.), 2010, From policies to results. Developing capacities for country monitoring and evaluation systems, viewed from http://mymande.org/sites/default/files/From_Policy_To_Results.zip
Segone, M., 2014, Voluntary Organizations for Professional Evaluation (VOPEs) learning from Africa, Americas, Asia, Australasia, Europe and Middle East, UNICEF, Geneva.
Segone, M. & Rugh, J., 2013, ‘The growth and evolving capacities of VOPEs’, in Voluntary Organizations for Professional Evaluation (VOPEs) learning from Africa, Americas, Asia, Australasia, Europe and Middle East, UNICEF, Geneva, viewed 16 February 2017, from http://www.evalpartners.org/sites/default/files/UNICEF_NY_ECS_Book2_web.pdf
The New Institutional Economics, viewed 16 February 2017, from https://en.wikipedia.org/wiki/New_institutional_economics [16-02-2017]
Tullock, G., 1967, ‘The welfare costs of tariffs, monopolies, and theft’, Western Economic Journal 5(3), 224, viewed 16 February 2017, from http://cameroneconomics.com/tullock%201967.pdf
UNDP, 2009, Capacity development: A UNDP primer, UNDP, New York, 64 p.
UNICEF, 2013, Briefing Note Enhanced Programming and Results through Monitoring Results for Equity Systems (MoRES), viewed 20 October 2016, from http://www.unicef.org/about/employ/files/MoRES_Briefing_Note.pdf
Von Neumann, J. & Morgenstern, O., 1944, Theory of games and economic behaviour, Princeton University Press, Princeton, NJ.
Wikipedia, Core game (game theory), viewed 16 February 2017, from https://en.wikipedia.org/wiki/Core_(game_theory)
Wikipedia, Repeated Game, viewed 16 February 2017, from https://en.wikipedia.org/wiki/Repeated_game
Williamson, O., 1985, The economic institutions of capitalism: Firms, markets, relational contracting, Free Press, New York.
World Bank, 1998, Evaluation capacity development in Africa: Selected proceedings from a seminar in Abidjan, OED proceedings, World Bank, Washington, DC.
1. U-Report is a free SMS social monitoring tool for community participation, designed to address issues that the population cares about. See https://nigeria.ureport.in/
2. A strategy is dominant if, regardless of what any other players do, the strategy earns a player a larger payoff than any other. Hence, a strategy is dominant if it is always better than any other strategy, for any profile of other players’ actions. For the prisoner dilemma, the dominant strategy is not to cooperate (defection).
3. Like the attorney in the Prisoner Dilemma example.
5. A new deal offered by the police, if we take the Prisoner Dilemma analogy: confess or not confess.
6. More can be understood on coalition game here: https://en.wikipedia.org/wiki/Cooperative_game_theory