Description of the subject of the evaluation.
The project Supporting immediate needs and enhancing community self-sufficiency in response to new and chronic conflicts in CAR, is a one-year multi-sectoral project funded by BHA (USAID). Initially, this project should be implemented both in Ouaka (Kouango-Grimari axis – the same areas of the previous FFP Project including Pissigiri and Sioua) and Basse-Kotto prefectures. However, based on the context and difficulty accessing Basse-Kotto, a modification has been made in the project replacing Basse-Kotto by the Bianga axis (River sector). This project is thus built upon and complements previous activities in the Ouaka region.
This project aims to reduce the impact of recent and chronic conflict by meeting immediate humanitarian needs and enhancing community-based self-sufficiency and resiliency to future shocks. In other words, the project aims to alleviate the immediate and prolonged impacts of conflict through food provision, enhancing agricultural sustenance, community-centred nutrition and protection, and access to water. The project aims to reach 49,170 beneficiaries directly from 8,195 households with a holistic and integrated approach to meeting critical, immediate needs. It is designed to tackle three levels of intervention (in order of priority): i) Individual – meeting immediate and essential food security, water supply and inter-related needs (such as nutrition, hygiene and protection) ii) Social – creating self-sustaining community platforms to prevent and better respond to nutrition and protection risks (including referral pathways for severe cases) iii) Structural – constructing and rehabilitating water points for critical self-managed services. The project pursues three main outcomes: a) Targeted communities show enhanced food security and the capacity to enhance local production. b) Targeted communities reduce the risk of malnutrition and related health consequences. c) Targeted communities have enhanced the quantity and quality of water supply and targeted the following sectors:
- Food Assistance through direct food distributions: 50 days of food coverage, covering 50% (1.050 Kcal/pers) of total recommended calorific intake (2,100 Kcal/pers). This project was conducted using a blanket approach. This project has reached 8,195 households. Each household has received a food package of 67Kg, which consists of maize (15kg), cassava (15kg), rice (12kg), oil (5kg), beans (8kg), peanuts (6kg), sugar (3kg) and salt (3kg).
- Agriculture through distributing agricultural materials (seeds and tools) and agricultural training. Each of the 8,195 households targeted by the in-kind food distribution received seed packs (up to 5 varieties of 10g each) and tools to cultivate home gardens or support agricultural crop production. Distributions were accompanied by training and ongoing supervision provided by the RTAs (Relais Techniques Agricoles) trained and equipped by Concern to maximise the seeds through capacity development and coaching.
- Nutrition through training of health professionals, community-wide sensitization and the development of referral pathways focusing on Maternal Infant and Young Child Nutrition in Emergencies. Nutrition activities were focused on Maternal Infant and Young Child Nutrition in Emergencies through a community-led approach. 222 Community Health Workers (Relais Communautaires, RECOs) have been identified and trained to lead mass sensitization and support groups for pregnant and lactating women (PLW) and their partners. Sensitization will also focus on WaSH to encourage community members to adopt good hygiene practices that can save their life. Also, 24 Social and Behavioral Change (SBC) Committees were trained (one per village) to coordinate support groups and broader community dialogue exercises.
- Water, Sanitation, and Hygiene through enhanced water supply via water point construction and rehabilitation. This project has reached 11,500 beneficiaries through the construction (7) and rehabilitation (16) of 23 water points throughout both intervention zones. Improved water supply was provided complementary and synergistic work that will both reinforce and support other project activities. Under this sector, training was held for water management committees in terms of sustainability.
Evaluation objectives and scope
This evaluation aims to assess all project components, the level to which the project satisfies its targeted outcomes and establishes the key learning points for future interventions. This will be based on DAC criteria and the following specific objectives:
- Assess if the project has targeted the extreme poor and vulnerable people;
- Assess progress made towards the achievement of objectives, indicators and targets and the appropriateness of the targeting strategy used in the context;
- Asses to what extent activities adhered to planned implementation and achieved intended goals, purposes and outcomes.
- Support the organization’s commitment to accountability to donors, government, public, and beneficiaries through assessing good practices, gaps and adjusting the learnings in future projects;
- Assess the progress made by participants in regard to self-sustainability and resilience to future crises;
- Asses to which extent participants, community members and others were associated with programme design and implementation;
- Measure the performance of the intervention regarding the seven DAC criteria: coherence, relevance, efficiency, effectiveness, coverage, connectedness, and impact of the project according to a grading scale;
- Identify lessons learned and provide practical and innovative project options for assisting returnees in a sustainable manner based on integration and/or relocation;
- Provide recommendations on how the intervention could be improved in future.
The evaluation will focus on all engaged stakeholders (beneficiaries, governmental partners, implementation partners, local authorities…). This will cover the current BHA intervention period (August 2021 – October 2022 – including the No Cost Extension period). This evaluation will be conducted in all intervention areas (Kouango – Grimari axis and Bianga axis). Therefore, based on the security context during the evaluation period, the more accessible areas might be prioritized.
The evaluation will be a performance evaluation using mixed-methods approach. This will be led and facilitated by an external consultant. The evaluation will focus on all project components and activities, the way they have been implemented, the performance of the project and the level of involvement of stakeholders. Learning and improvement will be the two key focuses of this evaluation.
The evaluation questions will be based on the DAC criteria based on Relevance, Effectiveness, Impact and Sustainability and Learning. The following questions will guide the consultant during the process:
Relevance: Is the intervention doing the right things?
- Were interventions appropriate and effective for the target group based on their needs?
- Which target groups and individuals were reached by the interventions?
- How effective was the targeting approach in achieving the project goal?
Effectiveness: Is the intervention achieving its objectives?
- To what extent do the project’s interventions appear to have achieved their intended outputs and outcomes?
- To what extent did the project help prevent individuals and households from adopting negative coping strategies such as selling productive assets?
- What are the major factors influencing the achievement or non-achievement of the objectives?
- How has management adapted the project design or implementation based on monitoring information and feedback from the target population?
Impact: What difference does the intervention make?
- What changes—expected and unexpected, positive and negative—were experienced by the targeted beneficiaries and other stakeholders?
- How have the project interventions impacted differently on men and women, boys and girls, people with disabilities (and other vulnerable groups as identified) in the project area?
Sustainability and learning: Will the benefits last and what are key learnings from this project?
- To what extent did the project take advantage of other USG and non-USG investments in the same target areas to facilitate linkages with complementary services, layering with earlier investments, and implementing an exit strategy?
- Was the project able to end operations at the close of the award without causing significant disruptions in the targeted communities?
- How might we do things better in the future? Which findings may have relevance for future programming or for other similar initiatives elsewhere?
- What were the key learnings from this project (success factors, best practices, lessons learned)?
In addition to the DAC criteria mentioned above, the evaluation should analyze in depth the following cross-cutting aspects by demonstrating to what extent they have been taken into account in the project and/or how the intervention logic as well as the activities of the project, have integrated the following cross-cutting components:
- Sensitivity to conflicts;
- Adaptation to climate change;
The final evaluation will be a performance evaluation which will use a mixed-method approach based on the project’s logical framework and taking into account the proposed evaluative questions in the section. The consultant will submit a detailed evaluation methodology to be validated by CONCERN’s technical committee. This will include necessarily the following points:
- Project documentation review. The consultant will have to review all existing documentations regarding the project for instance: contextual analysis report, project document, periodic reports, reports of previous evaluations and any other document that can facilitate his/her understanding and knowledge of the project and the mandate assigned to him. This includes also briefing with Concern staff.
- Sample approach and size: The consultant will specify clearly in his/her methodology, all the information related to the sampling, the number of individual interviews, key informants and focus groups to be carried out. For the quantitative part, the population size will be divided in two cohorts (cohort 1: beneficiaries on the Kouango-Grimari axis and cohort 2: beneficiaries on the Bianga axis). The sample should be representative of the whole population of beneficiaries meeting the following characteristics e= 5% (error of margin); Z= 95% (confidence level). Also, a clear sampling plan for qualitative study will be proposed by the consultant including a range of different stakeholders from different geographic areas
- Data collection and tools: For data collection, a final mixed-methods approach which consists of both quantitative and qualitative data collection will be used. This will integrate a comparison of baseline and endline quantitative data as well as a qualitative study. The quantitative part will be based on the logical framework of the narrative document that the evaluator will use to measure the results and effect of this intervention on the direct and/or indirect beneficiaries and compare the outcomes from baseline to end line. The qualitative part will consist of conducting focus groups (FGDs) and semi-structured interviews with the beneficiaries, project team, community members, community leaders, local authorities, Health care staff, RECOs, and other stakeholders (local, national, governmental) to address specific key standard evaluation questions including questions about the following cross-cutting issues and themes: gender/equality; sensitivity to conflicts; protection and climate change. The consultant will use CONCERN’s existing tools and develop other pertinent tools to gather information that will help to better interpret and to triangulate the quantitative results. The detailed methodology will include these tools and a clear sampling methodology.
- Data analysis: The consultant will precise the types of analysis that he/she will make. For instance: statistical test, correlation analysis, level of significance, mean comparison, etc. These tools he/she will use to analyze both quantitative and qualitative data.
The consultant must provide several documents/outputs including the following:
- The technical proposition (maximum 20 pages). The latter will present the different stages of the evaluation process from planning to submission and presentation of the final report. This also includes a clear methodology including tools and a detailed project plan against a timeline.
- The financial proposition: The consultant will submit with the technical proposition a detailed budget including his/her daily fees, government tax, and all costs related to the evaluation.
- The inception report (after documents review)
- The final report of the evaluation (Paper version, Maximum 45 pages): This will include the results, conclusions, lessons learned and recommendations in terms of actions to be taken in the future in similar projects. The evaluation report must also contain an executive summary no more than three pages, and in its long version must analyze the DAC criteria and the six cross-cutting aspects mentioned above and provide an assessment of performance against each of them using the following rating scale: 5 – Outstanding performance; 4: Very good overall performance with few shortcomings; 3 – Good overall performance but with some minor shortcomings; 2: Generally acceptable performance but with some major shortcomings; 1: Barely acceptable performance with many major shortcomings; 0: Completely unacceptable performance or insufficient data to make an assessment.
- An electronic version of the final report, which will be shared internally with CONCERN and also with other partners operating in these sectors.
- A summary of the final report (maximum 3 pages): This will be used for communication and external publications on the CONCERN networks and/or other appropriate platforms;
- A PowerPoint presentation (maximum 25 slides): The consultant will elaborate and share a Powerpoint presentation in which methodology used and the key findings of the study, the learnings and the key recommendations will be included;
- The databases (raw and cleaned). At the end of the evaluation, the consultant will share with CONCERN team all the databases. These will be kept by CONCERN as evidence and used for other additional analyses if necessary.
- Any other deliverable according to the deadlines set by the calendar (minutes of meetings, visit report, etc.)
Line of Communication
The consultant will report directly to the Project Manager and the Program Director, but will work in collaboration with the Area Coordinator and the MEAL Manager. Also, for any technical or specific questions, the consultant will be in contact with the technical advisors in Dublin. Any important decision relating to the evaluation will be taken jointly with the CONCERN technical committee including the Program Director, Project Manager, and MEAL Manager Supported by technical advisors from Dublin.
The working conditions of the consultant are presented as follows:
- The consultant will work in the CONCERN office in Kouango office, with visits to the field ( Kouango-Grimari and Bianga axis) or to Bangui in accordance with the validated work plan;
- Travel costs (international and domestic flights, visa, COVID19 tests), per diem, accommodation and field travel will be provided in accordance with CONCERN RCA policy;
- Payment of the consultant fees will be made in two parts of 50% each. The first upon signature of the contract and the second after validation of the final report.
- The consultant must fulfil all formal CONCERN administrative requirements, such as signing the Project Participant Protection Policy (4Ps), code of conduct, child protection policy, per diem, …;
- The consultant must have his/her own laptop to carry out his/her work;
- As a rule, the consultant must make sure to have health and repatriation insurance and is responsible for its payment;
- The consultant must comply with the CAR security policy and security procedures in the country. Any breach of this obligation will result in the immediate termination of the contract.
Composition, skills and experience of the consultant
- At least a master’sgree in Statistics, social sciences, nutrition, biostatistics or related fields;
- Five (5) to ten (10) years of experience and expertise in project development, project evaluation (at least 3 years);
- Proven expertise in final evaluation of project/project using a mixed methodology (quantitative and qualitative);
- Experience working in difficult and rural areas, under-developed countries; previous experience in CAR is desirable;
- Proven Expertise in manipulating statistical software for quantitative data management such as SPSS, R, Stata, etc.
- Strong analytical skills;
- Good knowledge of food security, nutrition and/or WaSH programming
- Fluency in both spoken and written French and English (and if not English, he/she will arrange translation of all outputs)
Plan for Evaluation implementation
The consultant will propose a detailed calendar considering the following elements: document review, field visits, data collection and analysis, debriefing, and final submission of the evaluation report.
- Travel to Bangui
- Brief meeting (Skype if need be) with Concern staff
- Review of relevant documents
- Develop the evaluation framework
- Develop the data collection checklist
- Prepare inception report with a detailed schedule
- Travel to the Kouango-Grimari axis
- Recruitment of enumerators
- Training for enumerators
- Data collection (HH Surveys, FGD, KII, … with gender, disability and age disaggregation where possible)
- Travel to the Wakini-Zangba axis
- Data collection (HH Surveys, FGD, KII, … with gender and age disaggregation where possible)
- Travel to Bangui
- Data analysis, prepare and submit first preliminary results to Concern for feedback
- Travel to his/her country
Week 5 and 6
- Data analysis, prepare and submit a first draft report to Concern for feedback
- Produce the final report incorporating all the feedback from Concern staff
- Report submission to Concern and final restitution
Applicants are invited to submit a complete proposition (technical and financial) to provide the services described above. They must also provide information about their abilities and experience, demonstrating that they are qualified to carry out this work. (CVs, professional references, copies and/or links of evaluations already carried out, etc.).
Applicants must send their applications to the following email addresses: [email protected] ; [email protected] on October 15th 2022 16h00 mentioning the project’s title: « Supporting immediate needs and enhancing community self-sufficiency in response to new and chronic conflicts in CAR». Any late application will not be considered.
The application must include the following documents:
- Cover Letter,
- Curriculum Vitae including professional references
- Detailed technical proposition
- Detailed financial proposition (budget)
- At least an example of previous conducted evaluation reports