Consultancy: Ex-post evaluation of 4 BMZ funded projects in Kenya 261 views1 applications

1. BACKGOUND AND RATIONALE

arche noVa (AN) is a German NGO (www.arche-nova.org) working in rapid emergency relief, rehabilitation & reconstruction and long term development in 14 countries all over the world.

In Kenya since 2012, AN has been collaborating with Africa Sand Dam Foundation (ASDF), a Kenyan NGO. The two partners have till now realized five development projects in a continuous way with funds of the German Cooperation spread out in three dry land counties of South Eastern Kenya: Makueni, Machakos and Kitui.

The centre of ASDF and AN development approach is the empowerment of community organizations called Self Help Groups (SHG) with practical knowledge, skills and low-tech infrastructure in order to attain sufficient water, food and ability to adapt to climate change. The SHGs are not established by ASDF and AN but were registered and existed before the start of the projects. The groups are a continuation of a tradition of community members coming together to address common interests as access to water, ‘merry-go-round’ (community led savings and credit groups), agricultural production and land terracing community groups.

ASDF has therefore been supporting registered Self-Help Groups (SHGs) to harvest and conserve water through construction of sand dams, rock catchments and water tanks in schools, promote climate smart agriculture and soil conservation. They provide knowledge and capacity development opportunities to small scale farmers in collaboration with County Governments, Kenyan research organizations and international NGOs like arche noVa.

Since 2010 ASDF has assisted over 201 SGHs (currently ASDF is working with 80 SHGs), reaching till now 16,740 small farmers from 2,790 households, constructed 438 sand dams, 300 shallow wells, 179 school water tanks, 11 rock catchments, planted 575,892 trees and dug 511.236 metres of terraces.

The evaluation will focus on four interventions that were implemented by ASDF in collaboration with AN. Three of the projects are already finished and one is ongoing, but under completion in November 2019.

In almost 7 years (2013-2019), the overall timing of the four projects under evaluation, 34 sand dams and 2 rock water catchments were built by 19 SHGs. Furthermore, ASDF and AN assisted them providing trainings on practicing soil and water conservation, sustainable agriculture and agroforestry, planting drought tolerant crop varieties, establishing seed banks and utilizing energy saving stoves.

projects under evaluation

1. Improvement of the Water supply and the food security in Makueni County in the Eastern Kenya

01.12.2012 -30.11.2013

2.

Improvement of water supply and food security in Makueni and Machakos Counties in the Eastern Kenya

01.09.2014 – 30.09.2015

3.

Community Based Reforestation and Water Storage for Increasing Resilience in Makueni County, Kenya

01.01.2015 -31.12.2017

4.

Strengthening resilience and nutritional security of rural subsistence farmers in Makueni, Machakos and Kitui Counties, Kenya

01.11.2016 -31.10.2019

Detailed information about the projects under evaluation you will find here:

https://arche-nova.org/en/node/791

2. OBJECTIVES OF THE ASSIGNMENT

The main objective of this assignment is:

· To conduct an evaluation of four projects implemented by ASDF in three sub counties- Machakos, Makueni and Kitui in the South Eastern region of Kenya.

The specific objectives are:

· To verify the relevance, effectiveness, efficiency, impact and sustainability of the four selected projects.

· To assess the situational, projects and impacts analysis extent to which the projects performed at output and outcome levels compared with the base lines submitted in the final evaluations realized by the project partners.

· To provide evidence-based knowledge on effectiveness of Community Led Development model that will be transferred to the target communities, the local authorities, the implementing partner, ASDF, the lead agency, arche noVa and the funding organization, BMZ.

· To provide practical and actionable recommendation on how to improve the quality of the interventions helpful in ongoing and plan for future projects design.

3. SCOPE OF THE WORK

To assess the extent to which the projects have performed at output and outcome levels compared with the base lines (when available), as reported in the internal final evaluations realized by ASDF and AN monitoring team and the final reports submitted to BMZ.

In particular, the consultant will collect data and will provide information on:

**Situation Analysis***,* the evaluation will provide a brief overview of critical elements of the broader societal and national context that contribute to shaping the 4 projects. The organizational context and role of the local partner and target groups will be sketched out. The changed situation in the projects’ areas at the beginning of the interventions till today will be described.

Projects Analysis will provide information of the overall performance and results of the projects to date. Changes or amendments in the projects from the original project plans (locations, measures, number of participants and/or target groups) will be taken in consideration. Participation and ownership of target groups will be analysed (gender empowerment, cooperation and involvement of other stakeholders, influence of external factors as climate change, social and political aspects).

Impact Analysis, special attention will then provide on projects’ impact on the different needs of women, men, boys, girls and vulnerable individuals.

To look for a long-term social impact of the partners’ Community Led Development Approach. This is intended to analyse how the inputs and capacity development opportunities provided to the target groups through the projects changed the lives of the communities and if these changes were sustainable. The questions of the evaluation will not only focus on the outputs, outcomes and impact of the specific projects, but will also analyse the Community Led Development Approach implemented through these projects.

4. PROCESS

The methodological approach will include a combination of data collection, direct observation and a discussion with key project staff, stakeholders, direct beneficiaries and communities. The process of collection, analysis and dissemination of data and information will be 3 months (October-December 2019). The methodology and steps to be implemented to achieve the desired results will be as follows:

Preparatory and briefing phase

· Briefing with the project managers and program coordinators on definition and expectations from the evaluation, through elaboration of key questions.

· Review of project documents which includes project proposals, baselines and end-lines, narrative and financial reports, monitoring and key study reports.

· Designing of the methodology and data collection system.

Development and presentation of an inception report including work and mission plans, questionnaire specifying which geographical areas and target groups will be covered, with which method qualitative and / or qualitative information (questionnaires, focus group discussion, interviews, observation etc.) will be collected during the evaluation and gap assessment. The consultant will prepare relevant tools to collect relevant data. The documents will be shared and discussed with AN and ASDF before his/her mission to the field. Their feedback will be included by the consultant.

Field work

· Interviews with ASDF and AN staff- projects managers, field managers and outreach officers.

· Field observations of some of the project physical set ups- sand dams, shallow wells, demo plots, etc.

· Qualitative and quantitative survey of the target direct beneficiaries. It is envisioned that the survey will involve at most 10% of each target group (women, men, vulnerable persons etc.) of the community members participating in the projects.

· Focus group discussions, key informant interviews with a sample of the beneficiaries, host community members and other stakeholders; Ministry of Agriculture, Ministry of environment and natural resources, County government officials in charge of water, sanitation environment and climate change and organizations involved implementation of similar projects.

To ensure transparency and freedom of expression, FGD and key informant interviews will not involve employees of the local sponsor or AN. All collected data is treated confidentially and anonymously and does not allow any conclusions to be drawn about individual persons.

Report writing

· Production of draft report containing systematized information and draft recommendations.

· Discussion of the findings with key ASDF and AN staff.

· Validation meeting with key informants interviewed in the community.

· Submission of the final evaluation report including practical and actionable recommendation how to improve the quality of the intervention and that will be helpful in future project design; a file of good quality pictures and a contact list of all the people met during the evaluation will be attached.

5. OUTPUTS AND DELIVERABLES

The evaluation deliverable will consist of the following;

· Final evaluation report will be prepared by the external consultant in English, in a user-friendly format. The document will be discussed and confirmed with AN and ASDF then presented to the BMZ. The report will include a detailed report on outputs achieved and their indicators (quantitative and qualitative); sustainability, efficiency, effectiveness of the intervention.

The evaluation report will also include practical and actionable recommendation on how to improve the quality of the interventions helpful in future project design.

· A file of good quality pictures (technical and social pictures).

· A contact list of the principal stakeholders met during the evaluation.

At the end of the evaluation ASDF and AN will present the main results of the evaluation to the

groups and other stakeholders involved in a validation meeting. The event will be organized in a

location in the target area at the presence of representative of each SHG, local authorities, donor

and other organizations of the civil society.

A panel will be organized to discuss about social impact and how to empower community to lead

their own development.

6. MAIN QUESTIONS SUGGESTED

The report will answer to the following key questions, the questions are not exclusive, they only serve as a guide:

Relevance

Regarding the assessment of the relevance and appropriateness of the interventions with beneficiaries’ requirements, county needs, national priorities and partners’ and donor’s policies the questions suggested are:

• Are the project objectives consistent with the County Integrated Development Plans, priorities and policies at local and national level?
• Are the project outcomes relevant to the priority needs of the target groups, including women, youth?

Effectiveness (results achieved)

The consultant will assess the achievement of projects results, coverage of project areas and project coordination mechanisms. The questions suggested could be:

• Have the interventions achieved or is expected to attain the projects outcomes and objectives?
• Did the projects only achieve intended results and objectives, or were there unintentional, negative or even counterproductive results, which may also have longer-term effects on the target groups?
• Did the projects successfully reach the intended groups; male and female farmers, including their families?
• Have the projects missed any group in the community, especially, any marginalized group?
• Were the principles of quality, inclusiveness, protection, environmental sustainability, gender equality, applied in any activity of the projects?
• Was the duration of the interventions long enough to support the target community to improve their lives and become independent from external support?
• How should the quality of the technical planning, management and implementation of the projects be assessed?
• Which measures were particularly effective?

Efficiency (activity realized)

The consultant will assess how economically resources/inputs (funds, expertise, time, etc.) were converted to results. The questions suggested could be:

• Has the financial contribution of the target SHGs to the project been adequate compared to their resources and income?
• Have the resources of the project been spent adequately to ensure quality of service provision and provide enough knowledge and information to the target groups for a sustainable change? Were there project areas or projects in which this succeeded very well? Were there project areas or projects in which this was less successful?

Overall development impact

Assessment of impact/outcomes of project activities

• To what extent has the project achieved the intended impacts and has there been any additional impacts?
• To what extent and how did the projects build the capacity of civil society?
• Did the partners, AN and ASDF support development of policies by County and/or National government institutions?
• To what extent is the capacity of SHGs to be strong organization supporting the needs of their and other communities?
• To what extent have the projects enabled women to have a greater voice in the community and involvement in decision making?
• Did the projects have a model character for improving water supply and food security and strengthening resilience in the region? Did you exert a broad impact and multiplier effects beyond the planned project framework, for example by assuming project components elsewhere or initiating similar projects by stakeholders?

Sustainability

Assessment of sustainability of projects interventions, specifically on continuation of intervention initiated by the projects

• How many groups are actively continuing to use the resources and skills provided by the interventions?
• What is the level of independence of the targeted SHGs from external aid?
• Are the targeted SHGs able to maintain and develop the achievements of the projects Are the target SHGs supported by the government officers to maintain and improve the achievement of the intervention?
• Does the local partner have the technical capacity to fundraise and manage similar projects without support of arche noVa?

7. CONSULTANT PROFILE

• The lead consultant should have a minimum of ten years’ experience of working in the development context

· Good knowledge of the country, national politics and structures and work experience in African countries

• Demonstrated experience with planning and implementing project evaluations
• Knowledge of community mobilization and participatory approaches
• Proven gender awareness and sensitivity
• Proven experience with Theory of Change and other strategic planning formats
• Experiences in qualitative and quantitative participatory M&E methods ideally with international and local NGOs
• Experience in developing tools for participatory data collection and analysis
• Experience of developing practical guidance documents for implementing staff
• Experience with participatory data analysis and data presentation
• Excellent English writing and orals skills
• Independence from AN and ASDF
• Knowledge of local language (enumerators)
• Ability to respond to comments and questions in a timely, appropriate manner
• Ability to write high quality, clear, concise reports in English
• Knowledge on WASH programs is an advantage

· Familiarity with the geographical area is an advantage

8. TIMEFRAME

The evaluation is expected to take place between October-December 2019. The consultant will receive all relevant documents from AN and ASDF. ASDF and AN will support the consultant to ensure the related administrative and logistics arrangements are met among which accommodation, transport and schedule the appointments.

Detailed information about the timeframe you will find here:

https://arche-nova.org/en/node/791