Original Research

Stretching between learning and accountability: Experiences of South African non-governmental organisations

Dzvinka Kachur, Sue Soal, Rubert Van Blerk

Received: 10 Apr. 2014; Accepted: 31 Mar. 2015; Published: 10 June 2016

Copyright: © 2016. The Author(s). Licensee: AOSIS.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

This article contributes to knowledge around organisational learning in relation to Monitoring and Evaluation (M&E) processes of non-governmental organisations (NGOs).

While learning and accountability are two fundamental purposes of M&E, in practice NGOs often perceive donor accountability as the only function of their organisation’s M&E system. Learning through meaningful monitoring of actions is a necessary process to satisfy effective functioning of organisations working on social change.

The article is based on action research, which included qualitative methods such as case studies, focus group interview(s), semi-structured interviews and questionnaires with participating organisations from an 18-month Monitoring and Evaluation Capacity Development Programme of the Community Development Resource Association, which involved nine South African NGOs. Organisations benefited from the programme through peer learning, consulting and research linked to the programme.

The research revealed that NGOs have a perception of M&E as an accountability procedure that has been imposed on them by donor communities and are resistant to it as well as to rigid data collection. Organisations require specific capacities to realise their power and to be able to negotiate more developmental approaches to M&E within their organisations and with donors. The model is suggested that fosters learning in M&E systems and includes two interlinked processes: self-awareness (a sense of core organisational values and intuitive ability) and awareness about the outside world and the effects of organisations’ work.

The model for M&E systems can be explored further and help those organisations who are working towards better balance between learning and accountability.

Introduction and context

At the end of last century, the demand for evidence-based policy making, attribution of results and proof of impact started dominating the discussion of development agencies. Nowadays, requirements for accountability using result-oriented Monitoring and Evaluation (M&E) and showing evidence of impact are mandatory requirements from most funders, with many also prescribing the ways in which evidence and results are shown (Eyben 2013). M&E has grown into a speciality field with its own rules, language, practice and specialists.

In theory, M&E ideally form part of our planned intentions within our programmes and projects and should be key processes in the working cycle of organisations and development practitioners, helping them to manage, support and improve their practice with consciousness and rigour (CDRA 2008; Dlamini 2006; Soal 2000; Taylor & Soal 2003). M&E should be composed of learning and accountability that are two interconnected processes. But in the developmental context, many practices show that M&E activities are driven by accountability interests of donor agencies over those of national governments (Carden 2009) and non-governmental organisations (NGOs) (Eyben 2013; Simister & Smith 2010; Soal 2001; Van Ongevalle et al. 2012).

Research in South Africa has shown that local NGOs are under a heavy donor accountability agenda which might shift the way NGOs do their work and, even more, it might inhibit organisational learning (Bornstein 2006; Mueller-Hirth 2012; Taylor & Soal 2003). Learning plays a crucial role for organisational functioning. The willingness to embrace new learning and investment in developing organisations’ capacity as ‘learning NGOs’ allows their accountability to improve (Bloch & Borges 2002) and to a large extent can define organisational success (Hailey & James 2002).

The critique of accountability-focused approaches to M&E is rapidly growing in the literature promoting alternative approaches, such as Developmental Evaluation (Patton 2011), Participatory Evaluation (Chouinard 2013) and ‘actor-focused’ Planning, Monitoring and Evaluation (Van Ongevalle et al. 2012). Alternative approaches are showing the importance of shifting from a sole focus on accountability to more participatory and learning-oriented M&E systems. These shifts in emphasis tend to overlap with developments in evaluation theory that recognise the challenge of complexity in social programming and the need to devise approaches that better work with complexity and emergence (Funnell & Rogers 2011). The field of theory-led evaluation offers a rich and full response to this challenge.

The shift from a results-based approach towards one that is more learning and participatory in emphasis is not possible without changes in the power balance between different stakeholders involved in developmental projects. As Simister and Smith write, taking different (political) interests, perspectives, demands as well as power dynamics and funding sources into account, it still remains a question whether M&E really can combine accountability and learning (2010:6–7). It is important that NGOs have the right capacity to resist, negotiate and change result-based requirements for accountability towards more developmental ones. As Rosalind Eyben states ‘the disciplinary power of artefacts is far from totalizing’ (2013:9), and thus, the response from the NGO depends on the culture and other capacities, including the capacity for reflexivity. While the literature suggests that there is opportunity to combine learning and accountability in one M&E system, there is little research showing how it is possible in practice.

The NGO ‘Community Development Resource Association’ (CDRA) initiated an M&E Organisational Capacity Development Programme (M&E Programme), which was launched in 2012. The goal of the programme was to promote learning as a necessary and integral part of M&E systems as well as to ensure better integration of the M&E systems with organisational and field practice. The CDRA has a very particular approach to development and its practice. It views organisations as complex, living systems that are in a constant state of movement along their individual paths of development. They support organisations working in the field of human development and social change to contribute more to innovative thinking and practices that facilitate transformation and development of the complex living systems that they are working with. M&E is seen as a vital organisational function, integral to improvement in both field and organisational practice, which ultimately translates into sustained and desired impact. M&E, in this view, is thus integral to the accomplishment of impact and is based on learning, not simply something that tracks and measures it (CDRA 2008; Kaplan 1999; Soal 2000).

The Programme combined five processes, viz. action learning, skills training, organisational accompaniment (OD), mentoring and action research into one coherent programme, offered to a group of organisations over a period of 18 months. In addition to the ‘modalities’ of action learning, skills development, OD accompaniment, mentoring and action research, the programme very intentionally formed a community of practice that was closely connected to its members and to the surrounding context. For instance, M&E was contextualised both historically and politically and use was made of case studies during the action learning. Skills such as observation, listening and questioning were practised in these contexts, both relying upon group trust and further deepening it. While a learning-oriented M&E system was promoted by CDRA in the programme, it was finally up to the organisation to decide if they were interested in pursuing such an approach.

The key for the programme was to make sure that participating organisations go through several stages:

1. (1) Understand the context of M&E: what are different M&E approaches? What is the politics of M&E?
2. (2) Through discussions and reflections with organisations’ teams to redefine what M&E means for the organisations
3. (3) To understand what improvements are needed that would still be in line with the organisation’s goals, values and beliefs
4. (4) To implement changes into the current M&E processes.

Nine South African NGOs participated in the Programme and two senior people who were either managers or M&E specialists represented each organisation.

The participating organisations formed a diverse group (Box 1). They ranged in size from 10 to 300 employed personnel and in reach of work from international lobbying to local city-based development.¹ Areas of intervention included education, environment, social justice, economic justice, community development, land rights, migration, justice, healing and reconciliation. Ways of working included community development, community organising, research, think-tanks, lobbying and advocacy. Approaches to organisational management and strategy also differed widely, ranging from differentiated structures with portfolios and traditional line management systems to flatter and more participatory approaches.

Research method and design

The action research conducted was a component of the M&E Programme and explored the understanding of learning-oriented M&E and key characteristics that, according to the NGOs, make their M&E system work for both organisational learning and accountability.

The action research approach was chosen as it is suggested for situations when the change is expected to happen at the time of the research, in particular as a research component of the change programmes (McNiff 2013). Action research also helps avoid biases caused by the interpretation of information by the researcher and permits practitioners to improve their understanding of their practice through reflection on that practice, proceeding directly to adjustments and improvements, depending on what has been seen in the process of reflection (Stringer 2013).

Because research was a component of the CDRA’s M&E Programme, it allowed data to be collected for the whole 18 months of the programme. In this time, organisations had set aside time to reflect and deepen their understanding of what ‘learning processes’ might mean and how they could be implemented in their organisation’s M&E systems and in the specific contexts in which they worked.

Prior to the research, all participants discussed the research methods and design and agreed to participate in the planned research activities. The document ‘CDRA’s approach to programme and research’ (March, 2013) was co-developed and approved by all participants. The data and findings produced by the research were presented to participants and discussed accordingly. Participants read through the draft reports written out of this article and agreed that their experiences were represented. All participants gave permissions to use the data generated during the Programme with their involvement.

The research methods included case studies, focus groups, semi-structured interviews, research journal, and questionnaires (see Figure 1). The action research cycles coincided with the learning events of the M&E Programme. After the learning events participants developed a set of tasks that they would like to implement in their organisations, which included sharing of learning from the programme.

FIGURE 1: Research methods and design.

The survey of participants (semi-structured interviews) explored existing challenges of organisations’ M&E systems before and after the Programme and allowed for the identification of challenges that NGOs were facing with their M&E systems. Expectations, actual implementation and the derived results were compared with an understanding of learning-oriented M&E and its associated practices. Surveys also contributed information to the development of case studies. Despite the detailed information from the surveys and case studies (the latter not being presented in this study), they formed the basis for insights into learning processes in organisations and to the conclusions reached here.

In order to explore the participants’ understanding of learning-oriented M&E more deeply, two sets of focus group interview(s) were conducted, which explored what learning-oriented M&E means and its associated practices. The two sets of focus group interview(s) were conducted 5 months apart, and the group of 20 practitioners was divided in to two groups to represent one person from each organisation in the group, every time. The responses were audio-recorded and then transcribed. After analysis of the focus group interview(s) data, this information was discussed with the participants. Based on the focus group interview(s) results, a questionnaire was developed to capture the key understandings of learning-oriented M&E and the kind of challenges that organisations were facing when attempting to implementing these.

Nine case studies were developed capturing changes in objectives, beliefs, practices and reflections of each NGOs representatives as they were progressing in developing their understanding of learning-oriented M&E and adjusting their M&E systems to what they believed is learning-oriented M&E.

Each of the learning events included a significant amount of group discussion and reflection around implementation of learning-oriented M&E practices. Information collected from the learning events was documented by two practitioners and a researcher in the research journal (collecting thoughts after each learning event and/or consultations with each of the NGOs) and contributed to deeper understanding of participants’ perceptions of M&E, learning and the politics of power in M&E. Information from research journals was also used to develop the case studies of individual organisations. During the programme, as case studies were unfolding, each NGO had an opportunity to revise their own information and to check its relevance.

Opening experiences: perceptions of monitoring and evaluation and resistance to specificity

Imposed reality – accountability

During the first 3-day learning event of the M&E Programme in January 2013, organisations stated key challenges that they are facing when working with their organisations’ M&E systems. Participants revealed that learning is crucial for organisational development and should be one of the key reasons for designing an M&E system. At the same time, while M&E is expected to be driven by the organisation itself, often in practice the organisation focuses on implementing its activities and M&E is reduced to being fulfilment of funding and donor requirements.

The participants stated that focus around requirements for evidence-based interventions makes them see M&E, including approaches that involve and even require learning, as an imposed reality. In a group discussion, the opinion dominated that the M&E at the moment was designed to satisfy donors’ requirement and does not correspond well with NGO’s organisational practice. Moreover, NGOs feel powerless to change it:

‘…to be politically correct. Our outcomes were imposed on us by the donor. And we were forced to implement top down very quantitative M&E system. This was against our values…’ (Programme Manager with over 10 years’ experience describing her difficult relationship with donor for community development project)

The underlying clash is between combining a more narrow scientific response (measuring outcomes, achieved states) with a developmental approach (describing and making visible the processes by which situations change over the time):

‘I receive a table with over 100 quantitative questions and I can say nothing about my project in those questions’. (Programme Manager with over 10 years’ experience)

Even more, the M&E requirements might aggravate organisations work:

‘We have invested a lot in building relationship with our partners in communities and then the way evaluation was conducted has completely compromised our achievements.’ (NGO Director with over 15 years’ experience)

When describing a difficult evaluation process led to compromised relationships with project beneficiaries, specific approaches to M&E are also required for organisations working in critical contexts:

‘The nature of our work, working in the environment of violence, does not allow us to provide evidence that the donor wants. Releasing details of tortured people would put them at risk. And the donor needs numbers, indicators and they are not interested in our learnings.’ (M&E Specialist and researcher with over 10 years of experience)

Specifically, three main reasons why organisations perceive the M&E system as being imposed were:

1. (1) the system of measuring the success of the project often does not correspond to the system of values that the organisation declares
2. (2) the definitions of project success by NGOs and donors often clash and
3. (3) M&E has a specific language and terminology to which organisations find it difficult to relate.

The attitude of seeing M&E as an imposed reality creates a reluctance to implement M&E as an integral part of organisational life. Thus, when objectives for improving their M&E systems were stated, the dominant idea was – how do I make M&E intervene least into my organisation’s activities:

‘We would like to learn how to carry on with our work and not to waste time on M&E.’ (Director of an NGO with over 20 years of experience)

Then, CDRA practitioners presented information on the politics of M&E and how M&E can be negotiated and that it could, and should, serve the needs of the organisation and be learning oriented. In a discussion that followed the presentation, participants agreed that if an M&E system arose out of their organisation’s need, it would be much more embraced:

‘I have learned that M&E is strongly connected to organisation development, values and is essential to our work.’(NGO director with over 10 years’ experience)

As result of the discussion, participants saw more potential benefit to their work than an M&E system simply for compliance and external accountability. As the Programme Manager with over 10 years’ experience stated:

‘I’ve made the shift on how I look at M&E from doing it for donor to doing it for me and my beneficiaries.’

And even more, felt the potential to discuss the M&E system proposed by the donors:

‘I have realised that M&E can be negotiated with funders if we have deep understanding and are ready to fight for partners’ rights’. (NGO director with over 15 years’ experience)

This was a significant shift in perspective, during the first seminar in January 2013, at an early phase in the programme.

Resistance to data – learning

While accountability is perceived as an ‘imposed reality’, M&E as learning is accepted as an essential need. To open thinking around M&E, participants were asked about the primary purpose of M&E for NGOs, and learning emerged as an important organisational need. During the discussion in the first seminar (January 2013), participants stated that:

‘Purpose of M&E is how to learn to be better’. (M&E Specialist with experience of working in different countries for international NGOs)

And

‘to work better as result of learning processes’ (Programme Manager with over 15 years’ experience)

The purpose of M&E was described by participants as a necessary process to learn about the environment in which the organisation is operating in as well as the importance of consistently improving their practice. This reflected an important shift in attitude towards M&E, but it presented a new difficulty: while the process of learning is associated with the practice of reflection, it seems to be disconnected from data collection and analysis. Organisations claim they know what the work is in the field and blame donors who impose their rigorous statistical and empirical requirements that clash with their understanding. However, it became obvious that very few organisations are focussing on data collection to the extent that allows them to account clearly for their time (to themselves or to others) or to detect changes in the field. Data collection as a side-by-side process of field practice for social transformation is often replaced by practitioners’ notion of knowing and understanding the field. While these sense impressions, memories and perceptions are essential sources of insight into situations (what we came to call self-awareness), they are only one part of what is required to work effectively and strategically, including with an M&E system that enables both learning and accountability. An important function of the M&E system is to provide data about the field work, both for learning and accountability purposes.

Initially, it was difficult for some participants to see the need for specific and detailed information – data – on the world in which they work, including baseline information of the situations they work with, and on their effects on that same world. We characterised this as a ‘resistance to specificity’, which may exist for a range of reasons, many of which are valid. However, this resistance is what inhibits M&E for both learning and accountability.

Understanding of learning-oriented monitoring and evaluation

During the focus groups, participants were asked to define how they understand what learning-oriented M&E is. It was described mainly as a combination of tools and practices that are characterised as:

Cyclical

‘When M&E is imposed it becomes an event, not a process that feeds into organisation so that organisation learns from the process. It is more of ticking boxes. The integration of M&E when it is cycle – that is learning-oriented M&E’. (NGO senior field manager with over 15 years’ grassroots experience)

Reflective/responsive on practice/work/strongly connected to practice

‘Learning occurs when we are in a strong connection with practice – we learn from what we do and then do our work better’ (M&E specialist with experience of working in different countries for international NGOs)

Participatory: involves staff members, beneficiaries and partners/networks

‘For organisations that are working with communities – communities are major source of information. Thus M&E is about ‘opening-up’ communities. This process can be more ‘excavating’ or more ‘partnering’. Good relationship with communities’ means good quality of information…’ (NGO senior field manager with over 10 years’ experience)

And

‘Learning-oriented M&E should lead to the organisational change which is possible if each individual understands it and is changing him/herself’ (Programme Manager with over 15 years’ experience)

Mirrors organisational values and objectives.

Results in new information or knowledge that is shared within an organisation

‘Learning-oriented M&E is about organisational learning. Practitioner is learning and reflecting as a part of a process but there is a need to ensure sharing between individuals in organisation, to ensure ‘system’ that manages [individual] learning’s into organisational learning’ (NGO Director with over 15 years’ experience)

Leads to personal and collective growth (capacitates everybody involved). Results in action/improves practice by learning how to do specific work better

‘Learning-oriented M&E should fit into what you do and help you to be better’. – (Director of an NGO with over 20 years’ experience)

Reveals the impact of work/supports self-accountability. Occurs only if safe space is created:

‘Learning-oriented M&E is about creating a space for people to think and reflect. And this can be done in the institutionalised manner when you are blocking some space and also that if some issues are arising you are creating space to make sure that the issue is discussed but that nobody is attacked.’ (M&E specialist and researcher with over 10 years’ of experience)

One of the participants has also recalled the story that the M&E specialist, when working with the field workers, has realised they had a number of ideas on how to improve the practice. When asked why they did not communicate those ideas to the management, it was said that there was no space where they can promote their learnings.

It requires willingness of individuals working in the organisation to learn and to be open for new experiences.

After these characteristics were presented to the group, participants were asked to read about different requirements needed for the learning to happen as suggested by Sue Soal (2010):

1. (1) Space – investing time and resources in having learning as an intentional and particular activity, not added to any other, or assumed to have happened, by virtue of any other.
2. (2) Rhythm – there is a need to connect to the external environment, which is continuously changing: the field of work, the challenges of context, the work processes and what we learn from them and structure in space to bring experience from the field into learning in the organisation.
3. (3) Champion – there is a need to have a power in organisation, not necessarily the leader of organisation, but a person or collective that holds learning process, and is clearly mandated – ‘empowered’ – to do so. It is important that members can see the value that learning brings.
4. (4) Approach – this requires an inquiry/listening mode; deliberate investment in building trust and team-based learning requiring social facilitation skills and the collective development of instruments. Continuous development of approach is as important as the holding of space and rhythm.
5. (5) Collegiality – a quality of relationship within the organisational team based on mutual respect and a sense of common purpose towards collectively achieving organisational goals.

In the discussion that followed after the reading, characteristics for organisational learning were regrouped into six conditions. It was concluded that leadership is the necessary element, while others are complementary.

1. (1) Leadership plays a crucial role for learning in organisations. There was a lot of discussion around leadership and learning and all participants agreed that leadership is one of the most influential factors and that in the heart of the learning organisation is a ‘learning leader’.
2. (2) Learning should be a recognised value in organisation and be part of organisational culture. For introducing learning in organisations, it is necessary to have learning amongst the organisational values. If an organisation did not see learning as a value, it would be unlikely for it to have successful learning models. The values of organisation form the foundation for the learning culture and are strongly linked to leadership in organisation, whether formal or informal.
3. (3) Distribution of power/relationships and a safe environment within the organisation also plays an important role. The processes within organisation should allow different members to present their ideas to leaders/managers and to have open and transparent, non-judgemental discussion.
4. (4) Organisation-wide – to be integrated into everyday activities and serve a strategic goal, the M&E system should be integrated into activities of every member of the organisation. This means that all members buy into the concept of an M&E system and have the necessary skills to implement the desired model. The buy-in should be developed in a way by which people are managed and communicated with and in the way information flows.
5. (5) Close contact and involvement of stakeholders and beneficiaries provides the necessary triangulation for ideas and learning emerging in the field.² While this sounds almost idealistic, involvement of partners and beneficiaries in understanding of organisations provides legitimisation for the learning and is the strongest recognition of results that organisations achieve in the field.
6. (6) Space for learning – learning within organisation requires dedicated time and space. It means that there is a need for a set-aside time within organisations where learning is regularly happening.

Going further: the contemporary trends and challenges that non-governmental organisations face in relation to accountability and learning

Even when understanding what is required for learning-oriented M&E to happen, its implementation in practice for organisations remains very challenging because of both internal and external factors.

Based on questionnaires (May, September 2013), key reasons have been stated as to why organisations themselves struggle to integrate learning-oriented M&E into organisational activities:

• Knowledge of M&E tools and language;
• Heavy load of everyday activities, which does not allow for reflective space; and
• Vision and understanding of what data are important to collect.

In addition, some organisations are working on different programmes that are not very strongly connected with one another. This implies a challenge for the organisation to learn from its diverse programmes as there seems to be little integration between different activities.

Besides internal factors, the outside environment plays an important role in the ability of organisations to implement their learning-oriented M&E system. According to participants, three main factors forced their organisations to change their M&E approach: (1) funding crisis that caused downsizing and organisational restructuring; (2) requirements for donor funding and (3) change of leadership.

The funding crises had a significant influence on the whole sector. Some larger organisations managed to avoid it for a short period of time, but as the situation did not improve, they had to adapt to new financial conditions through retrenchment of staff, programmes and rethinking their strategy. Of nine organisations participating in the programme, only one had not done restructuring within the previous year or was not currently going through it.

When organisations are going through changes, the M&E system can play an important role in shaping the strategy and prioritising key areas. The need for accountability is greater as organisations have to perform better to get the funding. At the same time, learning plays a crucial role for holding and guiding changes within organisations. As Reginald Revans states, the ability of organisations to learn should grow if the turbulence of the external environment is growing (Revans 1998). This is confirmed by the feedback from participants:

We had our learning events before, but it is only after the crisis hit us that we really started to concentrate around what we can see in the field and what results we can see from our work (NGO senior field manager with over 15 years’ grassroots experience recounting story of own organisation that had gone through significant reorganisation a year ago)

Learning-oriented M&E can also be used as a recovery process to redefine the strategy of the organisation that has gone through significant crisis:

‘Learning and working on our M&E system is serving as a recovery process for our organisation that have downsized and need to redefine a new purpose and strategy’.– (NGO Director with over 15 years’ experience that suffered staff reductions of over 30% during the previous year)

So, how should learning work?

With the changing perception of participating organisations, increasing willingness to embrace learning and defining key factors that support learning in organisation, the question about learning practices arose. How can learning be made to be a living and relevant part of organisational life, ensuring it gains, rather than loses, credibility as it unfolds? Or how should learning be practiced so that it is not only around accountability and does not become artificial and meaningless?

Learning processes

From the focus groups and following discussions, it was agreed that learning-oriented M&E systems require a combination of reflection and data collection:

‘We should agree and understand that learning M&E is a practice that allows people to reflect on information that they are receiving, generating, co-producing etc…’ – (M&E specialist with 5 years’ experience of working for a nation-wide NGO)

Another M&E specialist and researcher with over 10 years’ of experience shared her example:

‘When the facilitated conversation started around data, that is when the M&E started to be productive as the conversation became around what we want here to do, what are our assumptions about healing and how do we see the process. This is what learning-oriented M&E should really be about.’

So when an organisation has a data collection system in place and has regular reflection cycles in balance, then the learning-oriented M&E can work as an effective system that provides enough strength to the organisation itself, enabling it to be aware about its intentions, work, environment and changes.

Before the final learning event, practitioners and a researcher got together to read all the materials generated during the programme: questionnaires, focus groups, case studies and a research journal. After reading all the materials, two main issues were identified that came as key tasks for learning-oriented M&E: helping organisation to know who they are and understanding how the work of the organisation is related to the social change that is targeted by the goals of organisation.

Some participating organisations revealed that working on their M&E system during the programme and trying to bring a learning component into their work made them realise that they need to work more on the identification of their theory of change and objectives:

‘organisation and all people related to it should be very clear on what it is that they are expecting to achieve in beneficiaries, change on the ground. And if there is not very much clarity about the expected change then the opportunity and focus for learning gets distorted.’ – (NGO senior manager with 30 years’ Director and programme leader experience who realised that the mission and vision of organisation was not clear enough and was causing problem for rigour M&E system)

Others who had well-developed objectives were lacking a good connection with the outside world and so less able to track the changes that their work was leading to:

‘We believed that we have a very good M&E system and we understand what change we want to achieve and what we are doing. But we have to recognise that our connection with the outside environment and data collection about the impact of our work in the field is not a strong point. (Director of NGO with over 20 years’ experience that has a number of learning practices established)

So based on this, a theoretical framework that involves two linked processes was suggested. These processes are ‘self-awareness’ and ‘awareness of our work in the world’ (Figure 2). At the last learning event in April 2014, this theoretical framework was discussed and clarified with all participants. After the learning event, this description was presented to each participant in writing and jointly discussed by researcher and practitioners with representatives of each organisation. The two processes were finalised and are explained below.

FIGURE 2: Learning processes comprising organisational learning.

Self-awareness – means learning about the organisation itself (individuals, values, culture) and intentions of the organisation or, in other words, building organisational unity and coherence from the inside. Self-awareness allows the organisation to act in a consistent way and be aware of the intended changes that it wants to achieve, as well as the possible limitations to achieving these. It allows organisations to have understanding of their identity, be focussed on values underlying the organisation and have a high level of understanding of intentions. This self-awareness strengthens the organisation to resist distractions and pressuring requirements from the outside world, enabling it to engage with these and adapt according to its own values, rather than being purely reactive. Self-awareness also helps organisations to set a high standard for the quality of their work.

At the same time, being self-aware does not necessarily mean a strong connection to field practice and understanding of the changing world or awareness of whether the organisation needs to change in response to the changing world. The self-awareness process is focussed inside of the organisation and does not provide information about potential risks posed by the outside world (e.g. donor crisis, innovations of technology or practice, or even changes as a result of effective work, that then demand a change in strategy.)

Awareness of our work in the world – means knowing the object, issue or situation that your actions are targeting and being aware of the change that your actions are causing in the world.

There is a critical difference on what information is needed to create strong Self-awareness and Awareness of our work in the world. While Self-awareness can be generated based on internal reflections of an organisation’s members, strategic organisational learning and accountability for change requires permanent collection of data that is both preceded and followed by reflections.

With this understanding of learning-oriented M&E, it is obvious that internally reflective learning is not enough for accountability. Internal reflective learning practices should be enforced by rigorous data collection, which in turn enable strategic learning. In other words, neither learning, nor accountability, is possible without a clearly resolved relationship to, and approach to, specificity, data and data collection.

Discussion

NGOs are facing significant pressure for results-based M&E systems from the donor community. This leads to an imbalance between learning and accountability, which in theory should both be fundamental purposes of M&E. Learning is a necessary process to satisfy effective functioning of organisations in a complex and diverse world that requires tailored solutions. Seen in this way, understanding and use of learning processes for M&E would allow NGOs to implement them in practice and to be more successful.

One of the key reasons for the domination of ‘accountability’ in the developmental process is the politics between donor agencies and NGOs, which is uncovered in a number of publications, for example, by Rosalind Eyben (2006, 2008, 2013), INTRAC (Britton 2005) and CDRA (CDRA 2008; Taylor 2000; Van Blerk 2005). The research conducted shows that to improve the power balance, one of the first steps should be a change of NGOs’ perception of M&E as imposed reality and recognition of the necessity for rigour in data collection. When organisations understand their own M&E system and are confident in data collection processes, they can more effectively and confidently negotiate these issues with donors.

The ability of NGOs to design the learning-oriented M&E depends on different capacities of NGOs. The recent study jointly commissioned by Comic Relief, The Department for International Development (DFID), Big Lottery Fund, Network of International Development Organisations in Scotland (NIDOS) and Bond shows that NGOs studied ‘do take monitoring, evaluation and learning very seriously and make considerable investments in it as they see it as a means to improve their work and that of their partners’ (ITAD & Consulting 2014). It also shows that for most NGOs, their MEL systems are supporting them in making day-to-day project management decisions and strategic management and learning.

In this article, we did not discuss different forms of accountability – downward, upward or horizontal – rather, we united it under the general term of ‘accountability’ in order to stress the importance of the other purpose of M&E – learning. But it is important to understand that donors, or ‘upward’ accountability, are only one part of an accountability system (OECD 2013).

This research defines learning organisations based on the participant’s understanding, which includes key characteristics of learning process in organisation: being cyclical, reflective on work, participatory, integrated with organisational values and objectives, develop capacities of members and stakeholders, improve practice, reveals the impact of work and requires safe space and willingness from individuals to learn. These characteristics are well correlated with the most popular definitions of learning organisations provided by Peter Senge and Bruce Britton. Senge defines it as organisation ‘where people continually expand their capacity to create the results they truly desire, where new and expansive patterns of thinking are nurtured, where collective aspiration is set free, and where people are continually learning to see the whole together’ (1990:3). And Britton suggests that learning organisation means ‘the intentional use of collective and individual learning processes to continuously transform organizational behaviour in a direction that is increasingly satisfying to its stakeholders’ (2005:8).

In the final list of characteristics for the learning organisation, participants put specific emphasis on the role of leadership in establishing learning processes and also on its participatory manner. The importance of involvement of all participants in learning process provides opportunity for changing the power balance in donor–NGO relationship.

Also, the culture of learning is one of the characteristics that play an important role in establishing learning organisations. It creates an environment that supports and encourages collective discovery, sharing and application of knowledge on a day-to-day basis, through an ongoing process of inquiry, feedback, reflection and change (Gill 2010). The participants’ views of relationship between the culture and leadership goes in line with the study of Hailey and James, who explored South Asian NGOs and shows that ‘an NGO’s ability to learn is dependent on its organisational culture and in particular the development of an internal culture of learning’ but conclude that this ‘learning culture’ derives primarily from the attitude of the leadership towards learning (Hailey & James 2002). In the discussion on whether leadership or culture of learning is more important, it was concluded by participants that leadership is critical and can lead to establishment of the learning culture, while learning culture can be destroyed if there is no support from the leadership.

The research also reemphasised a significant crisis that NGOs are going through because of financial challenges. This critical time can be used as an opportunity to implement change in the M&E system and rebuild NGO as a learning organisation that would be more resistant to external challenges. Organisations are naturally resistant to change and have ‘limited capacity to learn, adapt and continuously improve the quality of what they do’ (Fowler 1997:64). It means that for M&E to be most beneficial for the organisation, it requires structural transformation. The experience of many down-sized, unrecognisable and simply shut-down NGOs will attest to, if organisation is not transforming itself, then the environment will transform the organisation. The suggestion that M&E can play a role of recovering mechanism to redefine the organisation strategy as was suggested by one of the participating organisations is a tool that might need further exploration and research.

The model that was proposed by this study presents the learning-oriented M&E system as two linked processes: self-awareness and awareness of our work. The importance of self-awareness and awareness of the work should be both based on strong data collection process and reflections. Self-awareness means learning about the organisation itself (individuals, values, culture) and intentions of the organisation or, in other words, building organisational unity and coherence from the inside. And awareness of our work in the World means knowing the object, issue or situation that your actions are targeting and being aware about the change that your actions are causing in the world.

This model for learning organisation can be explored further and help those organisations who are working towards creating learning organisations in practice.

Among the models proposed by the learning organisation literature the most known are the energy flow model (Pedler et al. 1991) which suggests a blueprint for learning company, Senge’s model (Senge 1990); seven dimensions of the learning organisation (Watkins & Marsick 1993), learning organisation building blocks (Garvin et al. 2008) which include supportive learning environment, learning processes and reinforcing learning through leadership; and Learning Organization Atlas Framework (Santa et al. 2014). Each of these models suggests different conditions that are required for an organisation to be learning. The particularity of the model proposed in this research is that it clearly defines two different areas of focus for learning organisation: inside organisation and outside world. The key idea of the focus inside the organisation (self-awareness) is to identify any possible misunderstandings and incongruence between the values and strategy and to create the environment for individuals to clearly understand the values and strategy of the organisation.

Identifying possible incongruence between values, strategy and activities seems to be a critical part for NGOs existence. As Bloch and Borges (2002) state, the three reasons why NGOs are even more sensitive to incongruence than other organisations are the following: (1) values justify the very existence of NGOs; (2) the inconsistency between values and actions is rarely addressed by NGOs; (3) existing M&E tools (evaluation, monitoring, planning) do not help to identify this inconsistency. Thus, it is critical that a learning organisation, especially NGOs, permanently focus on those two levels when designing its learning processes.

Limitations of the study

The key limitations of this research are around the number of participating organisations, the selection process of organisations and the time duration for which the organisations were observed. Research is based on the experiences of nine participating NGOs who have applied themselves to work on learning-oriented M&E. Thus, research organisations did not include organisations who believe that their learning processes are well developed or those for whom learning processes are not appealing. Further research that would include more organisations and organisations working in different sectors would be beneficial. Also, organisations were observed only during the 18-month programme. To understand if the learning-oriented M&E can be sustainable and whether processes suggested are sufficient, it would be beneficial to extend the time of observation and to check if interventions that were implemented by organisations become sustainable.

Conclusion

The focus around requirements for evidence-based interventions makes NGOs see M&E as an imposed reality mainly because of specific terminology, revealing differences in definitions of success and/or different priorities for M&E between NGOs and funders. Both the perception of M&E as an imposed reality and the resistance to data inhibit opportunities for NGOs to use M&E as a learning opportunity. During the restructuring and transformation (survival crisis) of organisations, greater focus on learning would increase the success of organisations and improve their adaptability.

A number of requirements that support learning within M&E systems in organisations have been identified. These include leadership, values of learning, distribution of power, making M&E organisation-wide, involvement of stakeholders and creating space for learning.

M&E is an integral part of the organisation development process. Despite high pressure from the donor community around accountability in terms and frameworks that they can access easily, learning remains an essential element for success of NGO’s activities, including success in their abilities to account.

Understanding of M&E as an essential part of organisation development allows for focusing around the interests of organisation in M&E rather than simply fulfilling donors’ requirements.

To ensure changes in organisations towards a learning-oriented approach, it is important that organisations understand the purpose of M&E as integral to effective organisational life and that they pay attention to data collection as well as run effective processes of learning, including making provision for participation, integration and sense-making.

Leadership plays a critical role in effecting changes towards integrating learning in organisation. It is the vision of the leader, the opportunity to change the power between different groups inside the organisation and creation of space and processes for learning that can support learning within an organisation.

Despite learning-oriented M&E being perceived as something that would require additional time and resources, it mostly involves a qualitative shift in the point of view, seeking self-awareness and integrating this with data received from external monitoring of the world and the organisation’s work in the world. If structured and integrated within organisational practice, it becomes a natural part of core organisational functioning involving a different use of available funds and time, rather than an investment of additional resources.

Acknowledgements

This research would be impossible without the hard work and contributions of all participants and participating organisations: African Centre for Migration & Society, Afrika Tikkun, Black Sash Trust, Breadline Africa, Catholic Welfare Development, the Centre for the Study of Violence and Reconciliation Economic Justice Network, Environmental Monitoring Group and Surplus People Project.

The CDRA M&E Capacity Development Programme was implemented with funding from the Rolf-Stephen Nussbaum Foundation.

The authors declare that they have no financial or personal relationships which may have inappropriately influenced them in writing this article.

D.K. is responsible for research design, writing case studies and conducting questionnaires and majority of interviews. S.S., co-designer of research and facilitator of the CDRA M&E Programme, conducted some of the focus groups and interviews. R.v.B., co-designer and facilitator of the CDRA M&E Programme, conducted some of the focus groups and interviews. All authors were representing Community Development Resource Association at the time of this research.

References

Footnotes

1. CWD and SPP work locally; Black Sash, Afrika Tikkun and Breadline Africa are national organisations; ACMS and CSVR work on the African sub-continent and EMG and EJN work from international to local levels.

2. In CDRA’s approach, this process would in any event also be indistinguishable from good community fieldwork, which by its very nature demands feedback, interaction and co-creation of meaning.