Consultancy to Support External Evaluation of SERVE Project 69 views0 applications

Consultancy to Support External Evaluation of SERVE Project

Terms of Reference (ToR)

Concern Worldwide is an international non-governmental humanitarian organization dedicated to the reduction of suffering and working towards the ultimate elimination of extreme poverty in the world’s poorest countries

Background

Concern Worldwide is a non-governmental, international, humanitarian organization dedicated to reducing suffering and working towards eliminating extreme poverty in the world’s poorest countries. Concern has been working in Kenya since May 2002, implementing emergency response, health and nutrition, livelihoods and WASH, governance and advocacy, and education programs and projects.

With the support of the Bureau of Humanitarian Assistance (BHA), Concern has been providing multi-sectoral agriculture, food security, nutrition, and WASH response to vulnerable households in Marsabit and Turkana Counties since July 2022 under multiple awards, including: Drought Emergency Response for Food and Nutrition Security (DERF), Enhanced Drought Emergency Response for Food and Nutrition Security (E-DERF) and Sustained Emergency Response and Recovery for the Most Vulnerable Households in the ASAL Counties of Kenya (SERVE). The goal of the programs is to deliver effective, life-saving assistance to individuals and communities suffering from the cumulative effects of a three-year drought and the ongoing El Niño floods in Kenya’s ASAL counties.

Purpose of the Consultancy

The evaluation aims to determine how effectively the program has achieved its stated results and objectives outlined over the past three years of awards, while also exploring future options for providing sustainable support to program participants. The evaluation findings will be analyzed and compared against key reference points, including baseline surveys, end-line surveys, beneficiary assessments, routine monitoring data, and other relevant information gathered throughout the interventions. This comparative analysis will provide a comprehensive understanding of progress, trends, and any deviations from expected outcomes. Additionally, the evaluation will focus on identifying lessons learned from the program’s successes and challenges. These insights will be systematically documented and incorporated into the planning and design of future interventions. By highlighting best practices and areas for improvement, the findings will contribute to refining strategies and fostering innovative approaches to better meet the needs of program participants while ensuring long-term sustainability.

The proposed external evaluation will address the following key evaluation questions:

Relevance: Is the intervention doing the right thing?

1. Were interventions appropriate and effective for the target group based on their needs?
2. How effective was the targeting approach in achieving the emergency project goal?
3. How has management adapted the activity design or implementation based on monitoring information and feedback from the target population?

Coherence: How well does the intervention align with the needs?

1. To what extent did the activity consider gender equity, protection, age, physical and emotional challenges of the participants, and risks to participation in various interventions in activity design and implementation?

Effectiveness: Is the intervention achieving its objectives?

1. To what extent do the activity’s interventions appear to have achieved their intended outputs and outcomes?
2. Did the intervention have varying impacts on different groups (e.g., based on gender, age, ethnicity, or socioeconomic status)? If disparities were identified, what underlying factors, such as access barriers, cultural norms, or systemic inequities, contributed to these differences?
3. To what extent did the activity help prevent individuals and households from adopting negative coping strategies such as selling productive assets?

Efficiency: How well are resources being used?

1. To what extent have the activity’s interventions adhered to planned implementation schedules?
2. What was the level of efficiency and timely delivery of the goods or services?

Impact: What difference does intervention make?

1. What changes— expected and unexpected, positive and negative— did the targeted beneficiaries and other stakeholders experience?

Sustainability: Can post-crisis sustain the intervention’s benefits, and does it strengthen long-term resilience?

1. To what extent did the intervention contribute to strengthening the resilience of affected communities or individuals against future shocks or emergencies

A score will be assigned to each criterion using the following grading scale:

5 – Outstanding Performance

4 – Very good overall performance with a few shortcomings

3 – Good overall performance but with some minor shortcomings

2 – Generally acceptable performance but with some major shortcomings

1 – Barely acceptable performance with many major shortcomings

0 – Unacceptable performance or insufficient data to make an assessment.

Concerns’ cross-cutting approaches, including Conflict Sensitivity, Protection, Disaster Risk Reduction, and the Environment, will be considered through the evaluation, and the Evaluation Report will include a section covering these cross-cutting approaches, outlining which approaches were considered and how successfully they were addressed in the program.

Objectives and Specific Tasks to be Undertaken by the Consultant(s)

The evaluation will use a mixed methods approach, integrating both quantitative and qualitative data. The evaluation will be anchored on the OECD-DAC criteria as presented above and led by an external consultant/firm.

The evaluation methodology will encompass, but is not limited to, the following:

1. Review all quantitative data on program indicators, including a comparison of baseline and end-line data, and analyze this data about the evaluation questions outlined above. Additionally, the evaluators will examine performance monitoring data collected throughout the BHA awards to evaluate the systematic implementation of results-based monitoring within the awards. This analysis will be based on data collected by Concern throughout the awards; hence, no further quantitative data will be collected.
2. The external evaluators will lead the qualitative data collection to address the evaluation questions using Focus Group Discussions (FGDs) with program participants and Key Informant Interviews (KIIs) with staff and stakeholders. The qualitative approach will also explore issues identified in the quantitative review, focusing on the effectiveness of the FCRM system, the sustainability of food assistance, agriculture (particularly livestock), WASH, nutrition, and barriers faced by men, women, and marginalized groups. This process will consider societal structures, demographics, and gender and age factors to inform future program design.
3. The sampling approach for primary data collection will be non-probabilistic and purposive, aligning with its qualitative focus. This approach aims to capture a diverse range of perspectives on specific aspects of the implementation of the Awards. The evaluation team is encouraged to propose a sampling strategy with distinct samples for focus group discussions in each sectoral intervention (Food Assistance, Agriculture, Nutrition, and WASH). The strategy must ensure the inclusion of female participants and Marginalized groups. The team should determine a sample size sufficient to achieve saturation on the evaluation questions, based on their analysis of the project and scope of work. The evaluation team is encouraged to propose a purposive sampling approach for staff and stakeholder interviews to achieve saturation on the evaluation questions.

   Concern will fully support and facilitate the sampling process to ensure the effective collection of evaluation data.

4. Desk-based research/preparation

   The evaluation team will review program documents, including proposals, progress reports, monitoring records, and distribution data, alongside Concern’s quantitative data aligned with BHA indicators. They will also analyze reports from other humanitarian organizations and sector-specific documents, such as SMART Survey Reports, Rain Assessment Reports, National Drought Management Bulletins, Disease Surveillance Reports, and Kenya Health Demographic System reports for Food Assistance, WASH, Agriculture, and Nutrition sectors.

5. Field-based research

   In the field, Concern’s Program Managers and the MEAL Team will support the evaluation by coordinating interviews and discussions with program participants, non-participants, and staff through household interviews and focus group discussions (FGDs). Key informant interviews with stakeholders, including program participants, local authorities, humanitarian actors, and the line departments at the county level, i.e., the Department for Environment, Water and Natural Resources, Department for Agriculture, Fisheries and Livestock Development, and Department of Health. The external evaluators will be responsible for proposing a sampling methodology for the Concern.

After the fieldwork, time will be allocated for the evaluator to analyze and review the collected data, draft the report, and refine subsequent versions based on feedback from partners, Concern, and other peer reviewers. As part of Concern’s commitment to downward accountability, the organization will ensure that the evaluation findings are effectively disseminated to program participants and other stakeholders. This process will take place at multiple levels:

1. Community Level Dissemination:

   The findings will first be shared directly with program participants through community consultations conducted in all program locations. The program teams to ensure that the information is presented in a clear, accessible, and culturally appropriate manner will facilitate these sessions. This will take place for over one month. This approach ensures that participants are informed about the outcomes and have the opportunity to provide feedback.

2. County-Level Forum

   Concern will organize a dissemination forum at the county level to share the evaluation findings with key stakeholders. This forum will involve various county departments, including relevant technical teams, as well as Concern’s County program staff and local partners. The objective is to foster a collaborative discussion about the results, lessons learned, and implications for future programming within the county.

3. National-Level Forum

   Finally, Concern will convene a national-level forum to present the evaluation findings to a broader audience, including national stakeholders, program staff, relevant authorities, and the donor. This forum will provide an opportunity for strategic dialogue, policy alignment, and resource mobilization to ensure the sustainability of program outcomes and the integration of lessons learned into national-level planning.

4. Outputs

The evaluator will be fully responsible for the following:

1. A concise inception report outlining the data collection methodology (qualitative and quantitative), data collection tools (checklists and questionnaires), work plan, and submission and review timelines. The report will also detail the proposed sampling approach, including sample sizes and the planned number of focus group discussions and key informant interviews for each respondent category.
2. A first draft of the Evaluation Report with an executive summary and clear recommendations (complete fewer appendices) for comment from Concern Kenya within 1 week of concluding interviews.
3. A presentation of findings to the Concern Kenya country team upon completion of the draft Evaluation Report.
4. A full final draft of the Evaluation report, integrating the feedback received within one week of receiving consolidated feedback on the draft report.
5. The report, in English, should be 10-15 pages long without appendices and should be submitted in electronic format (Word or PDF) to the country program and include the following sections:
6. Executive Summary (maximum 2 pages)
7. Brief context and description of the intervention
8. Presentation of evaluation methodology and any limitations encountered
9. Presentation of main findings, conclusions, and recommendations using graphs, charts, and tables where appropriate
10. All information should be disaggregated as per the PRIS for the BHA award, including where permissible and logically presenting the disaggregation per location.
11. The analysis should combine quantitative data outcomes with identified issues to inform future strategies for addressing barriers in humanitarian interventions and enhancing program impact.
12. Recommendations emphasizing key lessons learned to enhance Concern’s future emergency multi-sector program planning, implementation, and management responses
13. Scoring against the extended DAC criteria
14. Annexes: including the Terms of Reference (ToR), a list of people and groups consulted, interview frameworks/questionnaires, data collection tools, a list of sites visited, abbreviations, and any maps, charts, or graphs used in the evaluation.

Final payment is dependent on the submission of a good-quality, well-written final report or completion of agreed outputs (as detailed in the TOR). In addition to methodology, findings, and discussion, the report should include a comprehensive executive summary and a section outlining clear and concise conclusions and recommendations. A digital copy of all reports will be required by Concern at the end of the piece of work.

Timeframe

The external evaluation process is expected to take 30 working days. This timeframe encompasses all tasks, including document review, field visits, debriefing, and the final submission of the evaluation report, between the 6th of May and the 15th of June 2025.

Milestone Schedule and Number of Days

• Briefing, background research, and development of tools and methodology-4days
• Drafting of the inception report-3days
• Review and approval of the inception report -3days
• Internal and External interviews -10days
• Analysis and compilation of draft evaluation report -5days
• Debrief/presentation of findings upon completion of the draft report-3days
• Complete final report -2days

Total Working Days -30 days

Lines of Communication

The external evaluator will report to the MEAL Advisor and work closely with the Program Managers in each County.

Essential and Desirable Experience/Qualifications

1. Master’s degree (preferred) or bachelor’s degree (minimum) in Development Studies/Social Sciences/Statistics or Applied Research Methods/Livelihoods or social work.
2. Minimum experience of 5 years conducting evaluations along USAID/ BHA OECD evaluation criteria, ideally leading an evaluation team, and experience in designing evaluation methodology/tools, data analysis
3. At least 5 years’ experience using impact evaluation tools and methodologies
4. At least 5 years’ experience working in development contexts in Arid and Semi-Arid (ASAL) areas in Kenya
5. In-depth knowledge and at least 5 years’ experience in using quantitative and qualitative research methods
6. Individuals or firms with a background and at least 5 years’ experience in research methods, Livelihoods/Nutrition/Social Work, or development studies
7. Experience in writing evaluation reports to a high standard, in English

• Remuneration/Payment

The fee will be paid in 4 installments against invoices issued by the Consultant as follows:

1. Inception Report

Submission and approval of a detailed work plan and methodology 10% of the total fee

2. Field Work Completed

Completion of fieldwork 30% of the total fee

3. Draft Report Submitted

Submission of draft report with preliminary findings 30% of the total fee

4. Final Report Approved

Submission and approval of the final report incorporating feedback 30% of the total fee

All payments will be processed subject to withholding taxes. (5% withholding tax & 16% VAT where applicable for residents); (20% withholding tax & 16% reverse VAT for non-residents) under the Laws of Kenya or applicable tax obligations as per the Consultant researcher’s country of residence.

Expenses not specified in this or any section of this agreement are the responsibility of the Consultant.

Security

It is a requirement that the consultant will comply with Kenya security policy and in-country security procedures. Failing to comply will result in immediate termination of contract.

Required documentation with your proposal:

1. Company / Consultant profile
2. Certificate of Incorporation (firms) or National ID card (individual)
3. Valid Tax Compliance Certificate
4. PIN Certificate
5. Certificate/reference information of previous undertakings of similar contracts with

NGOs/UN agencies.

1. Safeguarding Policy, Code of Conduct etc. or what are the necessary provisions have you put in place to keep everyone safe in the course of your engagement
2. Attach copy of Certificate of Good Conduct or proof that you have applied for the same.
3. A proposed daily rate calculated for the 30 days, inclusive of all costs. Should be presented in Kenya Shillings & ensure you indicate VAT and withholding tax where applicable.
4. A detailed technical proposal
5. Ghant chart workplan