External Consultant for Final Evaluation of Concern Worldwide’s UK Aid Match-funded Project: “Post Ebola Livelihoods and Food Security Recovery for Vu 87 views1 applications

Terms of Reference

External Consultant for Final Evaluation of Concern Worldwide’s UK Aid Match-funded Project: “Post Ebola Livelihoods and Food Security Recovery for Vulnerable Households in Tonkolili District, Sierra Leone”

1.0 Background

Concern Worldwide has been operational in Sierra Leone since 1996, primarily in emergency interventions and then moving to development programs from 2002. Concern’s vision is to eliminate poverty for the extreme poor within the country and is guided by a poverty analysis based on three dimensions; return on or lack of assets, inequality and risks and vulnerabilities. To address the roots of poverty based on this analysis the organization takes an integrated approach working where possible across sectors of Education, Health and Livelihoods in the same geographic localities and beneficiaries.

One of Concern’s operational areas is Tonkolili District, a predominantly rural area with an estimated population of 530,776 of whom approximately 74.1% are significantly affected by food insecurity due to the outbreak of Ebola. Farmers in Tonkolili district experienced drastic reduction in food production due to Ebola containment measures put in place by WHO and the Government of Sierra Leone in the past years. And this led to high unmet food needs and economic vulnerability.

With support from DFID, Concern has implemented a post Ebola livelihood project targeting forty eight Ebola affected communities in Tonkolili district from May 2016 to October 2019.

Project Overall Objective: Mitigate the impact of Ebola on food security and protect future livelihoods of the most vulnerable households in Tonkolili district.

Project Specific Objective: Vulnerable households in Tonkolili district have increased food production and higher incomes

In line with Concern Wordwide’s PM&E guide, the project has put in place the M&E plan, monitoring database, baseline evaluation, and midterm review data for the indicators listed in the log frame. As the project is now ending, the final evaluation is scheduled to take place in October and November 2019. This document describes the Final Evaluator’s Scope of Work (SOW) for the evaluation based in Tonkolili, Sierra Leone.

2.0 Project Description

2.1 Project Population

Tonkolili District is a predominantly rural area, in the center of Sierra Leone. The estimated population is 531,435.^[1] Tonkolili was selected for this intervention because at the time of proposal, it had the highest number of individuals living below the poverty line at 76.4%[2] compared to a country average of 52.9%. Of this, 35% were living on less than $0.50 per day, with a higher proportion of female headed households in this category. Tonkolili also had 11.6% of the country’s burden of Ebola Virus Disease (EVD) with 458 cases.[3] The district was already experiencing food security problems before Ebola, with an average hunger gap period of 6.5 months.[4]

Inequality issues, especially related to women’s access to land, control over resources, violence against women and participation in decision making negatively impact on food security as women undertake 70% of subsistence farming, accounting for 80% of food production.

The project targeted a total of 4,000 households / 24,000 direct beneficiaries (14,400 female; 9,600 male).

2.2 Project Strategy

The UK AID Match, “Post Ebola Livelihoods and Food Security Recovery for Vulnerable Households in Tonkolili District, Sierra Leone” was designed to address food security challenges for 4,000 vulnerable households (HHs) directly & indirectly were affected by Ebola in the district to have sustainably improved food security through improved quality and quantity of their own food production and can generate sufficient income to meet on-going household (HH) food and non-food needs. The project is implemented across the district targeting forty eight Ebola affected communities in ten chiefdoms: Yoni, Malal, Mara, Kunike, Kunike Barina, Mayepoh, Yele, Tane and Kolifa Mayossoh, Kholifa Mamuntha.[5]

There were four expected outputs:

1. Improved farming practises among trained HHs to increase food production in Tonkolili district.
2. Reduced post-harvest losses through better storage facilities and improved seed management
3. Enhanced income for farmers by improved processing, adding value and marketing of their crops.
4. Improved access to credit and saving facilities through support to Village Saving and Loan Associations (VSLA)

Project strategies included the major components of Farmers Field Schools (FFS), Village Savings and Loan Associations (VSLAs), developing Inland Valley Swamps (IVSs), market road rehabilitation, community store construction, provision of rice milling machines, and many others. Major planned outputs include men and women trained on and using improved farming practices and crop varieties, reduced post-harvest crop losses, increased net income for farmers, increased use of mechanical processing, farmers trained on marketing strategies, and improved access to and use of savings and credit. ​

2.3 Partners

The primary collaborative partners on this project were the Ministry of Agriculture and Forestry (MAF) and Tonkolili District Council in Tonkolili district. Concern also maintains collaborative relationships with Non-Governmental Organisations (NGOs) in order to coordinate activities.

3.0 Purpose of the Final Evaluation

The independent final evaluation report needs to be a substantial document that (a) answers all the elements of the Terms of Reference (ToR); (b) provides findings, conclusions, and recommendations that are based on robust and transparent evidence; and (c) where necessary supplements Concern’s own data with independent research.

The Final Evaluation (FE) should be a performance evaluation, but should be relevant to national level stakeholders as well as to advance policy dialogues and best practice in-country. The FE is also intended to contribute to findings on reducing food insecurity in a rural environment.

The FE deliverables will be used to influence dialogue at the local, national and international levels on issues of food security, livelihoods improvements, village savings and loans (VSLA), crop yield, and crop storage and pest control. The evaluation report may serve to contribute evidence and learning for policy and programming decisions. The FE will also contribute learning to Concern Worldwide’s Livelihoods programming. Therefore, it is important that the final evaluator considers the audiences listed below, when conducting the evaluation and writing the report.

The FE provides an opportunity for all project stakeholders to take stock of accomplishments to date and to listen to the beneficiaries at all levels, including farmers, market vendors, pregnant and lactating women, other community members and opinion leaders, government agencies, other organisations, and donors. The FE report may be used by the following audiences as a source of evidence to help inform decisions about future programme designs and policies:

• In-country partners at national, regional, and local levels (e.g., MAF and other relevant ministries and civil society, PVOs/NGOs, local organisations, and communities in project areas)
• DFID and other UK Aid Match grantees
• Concern Worldwide headquarters and other Concern country programmes
• Concern Sierra Leone to inform on-going and potential future programs.
• Inter-Agency Working Group (IAWG)
• The international development community

The final evaluation will be disseminated and made broadly accessible to various audiences and findings will contribute evidence relevant to global initiatives such as the Sustainable Development Goals (SDGs).

3.1 Key Objectives of the Evaluation

The evaluation has three explicit objectives:

1. To independently verify (and supplement where necessary), Concern’s record of achievement as reported through its annual reports and defined in the project logframe;
2. To assess the extent to which the project achieved against the DAC criteria (efficiency, effectiveness, relevance, sustainability, and impact);
3. To specify key learnings from this project that should contribute to design and implementation of future programming and/or policies.

4.0 Verification of Internal Reporting

The first task of the final evaluation is to verify grantee achievement. The record of achievement will be presented in past annual reports, a midterm evaluation, and progress against the project logframe. This exercise could include verifying information that was collected by Concern for reporting purposes and possibly supplementing this data with additional information collected through primary and secondary research.

Verifying results from the project logframe will begin to capture what the project has achieved. However, there will be other activities and results that occur outside of the logframe that may require examination in order to respond to the different evaluation questions. Verifying reporting will also necessarily include a review of the data and systems that were used to populate results.

5.0 Evaluation Questions

The evaluator will use existing data collected or compiled during the life of the project, as well as additional data collected during the evaluation to answer the following questions that focus on the implementation of the project.

The final evaluators and the evaluation team will use the data described in Section 5.0 to answer the following evaluation questions around the following topics:

1. The quality of, extent to which, and process through which interventions achieved the intended results and how this approach impacted beneficiaries.
2. Relevance: How suitable were the activities and outputs of the project to the intended impacts and effects?
3. To what extent did the grantee support achievement towards the SDGs, specifically off-track SDGs?
4. To what extent did the project target and reach the extreme poor and marginalised?
5. Was the project responsive to the needs of the target beneficiaries, including how these needs evolved over time?
   1. How did specific activities respond to the needs of target beneficiaries?
   2. To what extent did project design, activities, and inputs appropriately consider and address gender dynamics, differences, and discrimination?
   3. To what extent were the promoted crop varieties (specifically, rice) appropriate for the agoecological context (soil, climate, taste, storage capacity, etc.)?
   4. To what extent were the selections of the focus crops justified, and were those justifications relevant throughout the project period and locations?
   5. To what extent did farmers utilize the storage facilities and agro-processing machines provided? What were key barriers and enablers in the use of these resources?
6. Was the project monitoring and evaluation design and implementation appropriate for measuring progress, achievements, and impact?
7. Effectiveness: To what extent did the project increase food production and incomes in the target population and areas?
   1. To what extent did the project improve food security in targeted households and communities?
8. To what extent was there meaningful adoption of new techniques or new seed varieties? Why or why not?
9. To what extent did the project activities increase food production in target communities?
10. To what extent did the project activities increase food storage in target communities?
11. To what extent did the project activities improve access to and operationalization of markets for input and outputs of the crops promoted under the project?
12. To what extent did the project activities increase sales of and incomes from target agricultural outputs?
13. To what extent did the project succeed in extending effective Farmers Field School systems and practices?
14. To what extent are the results that are reported a fair and accurate record of achievement?
15. What evidence exists and to what degree does it demonstrate that change is attributable to project activities?
16. Is there evidence to support success of the project to its objectives and goals?
17. What is the quality of data to measure impact?
18. To what extent has the project delivered results that are value for money?
    1. How well did the project apply value for money principles of effectiveness, economy, and efficiency in relation to delivery of its outcomes?
    2. What has happened because of DFID funding that wouldn’t have otherwise happened?
19. To what extent was the monitoring and evaluation of the project effective?
    1. What evidence and to what extent has the project used learning to improve delivery?
20. What are the key drivers and barriers affecting the delivery of results for the project?
    1. To what extent was the project design, including the logical framework, appropriately suited to implement and measure progress towards objectives?
    2. What were the key strategies and factors, including management issues, that contributed to what worked and what did not work?
    3. What were the contextual factors such as sociocultural factors, gender, demographic factors, environmental characteristics, cultural norms, economy characteristics, and so forth that affected implementation and outcomes?
21. Efficiency
    1. To what extent did Concern deliver results on time and on budget against agreed plan?
    2. To what extent did the project understand cost drivers and manage these in relation to performance requirements?
    3. What were the processes and mechanisms by which changes in attitudes and behavior were affected through project activities?
    4. To what extent did the project design and implementation succeed in complementing (rather than duplicating) existing services and service providers?
22. Sustainability: Which if any components of the project are likely to be sustained? How likely it that the gains achieved under this project will be sustained and what resources are required to foster an environment that will promote sustaining these achievements?
    1. To what extent has the project leveraged additional resources (financial and in-kind) from other sources? What effect has this had on the scale, delivery or sustainability of activities?
    2. To what extent is there evidence that the benefits and achievements, delivered by the project, will be sustained after the project ends?
    3. To what extent did the project create an enabling environment for increasing food production and incomes? Which elements of the project were most meaningful in affecting the environment in which project participants operate?
    4. Which elements of the project are likely to be sustained and/or leveraged and why or why not?
    5. How feasible will it be for communities and project stakeholders to sustain gains?
    6. What knowledge and resource gaps still exist in this context that need to be addressed and by whom?
23. Impact: What intended and unintended consequences has the project initiated?
    1. To what extent and how has the project built the capacity of civil society?
    2. What were the links between capacity building of farmers and food vendors, behavior changes, and outcomes?
    3. What effects has the project had, if any, beyond the directly targeted beneficiaries and communities?
    4. To what extent did the project beneficiaries receive support that they would have otherwise not received?
    5. How do intended beneficiaries view the project?
    6. What were the most significant change that the project had on the extreme poor and marginalised?
    7. What are the perceptions of beneficiaries and other stakeholders, including district-level officials, on the project and any change it has or has not affected? How do these perceptions differ based on characteristics such as age, gender, and social status?
    8. To what extent and how has the project affected people in ways that were not originally intended?
    9. What unintended consequences, positive or negative, did the project have on the market system and beneficiaries?
    10. Who were the people, groups, and/or entities affected unintentionally?

6.0 Key Lessons and Recommendations

The FE should make clear key lessons and recommendations on the scalability and transferability of best practices from this project to ongoing and future programming and/or policies in the same or similar contexts:

Key lessons and recommendations for programming: What are the key lessons and recommendations to take for improving food security programming?

1. What elements of this project, if any, have potential for scalability in the current contexts and why or why not?
2. Which elements of this project, if any, have potential for transferability to other contexts and why or why not?
3. What evidence, if any, did the project generate in support of integrating components of education, livelihoods, or other development sectors? What components or types of components are recommended and why?

Key lessons and recommendations for policy: What are the key lessons and recommendations to take for advocacy to government strategy and policy makers?

1. What results or learnings from this project, if any, are not reflected or not reflected well in current policies or strategies?

7.0 Evaluation Methodology

The consultants(s) (or consulting firm) commissioned to conduct the final evaluation and Concern are jointly responsible for choosing the methods that are the most appropriate for demonstrating impact. Evaluation methods should be rigorous yet at all times proportionate and appropriate to the context of the project intervention. Where possible, the evaluator(s) are encouraged to triangulate data sources so that findings are as robust as possible.

The final evaluator will lead a participatory evaluation. This means the evaluator will involve the relevant government agencies, project staff, project participants, and community representatives to make meaningful contributions to the FE. The evaluation will utilize a mixed-methods approach incorporating both quantitative and qualitative data, as well as the review of secondary data sources. The evaluator, with support from Concern’s Livelihoods Manager and National M&E Manager, will furnish a written methodology for the final evaluation (including the proposed number of key informant interviews, focus groups discussions, observations, and locations) and may be shared with project stakeholders for comment before the evaluation commences. Concern will facilitate sharing the methodology with project stakeholders in consultation with the evaluator.

Whenever possible, the assessment will also include observations of ongoing livelihoods activities, such as VSLA sessions, Community Conversations, operation of milling or cassava processing machines, and operation of community seed stores. This will involve site visits to implementation areas in Tonkolili district. The evaluation report will include a discussion of the methodological limitations of the evaluation.

Secondary Data

The evaluator will review project reports (Detailed Implementation Plan (DIP); annual reports; survey results; baseline and midterm review reports; VSLA tracking data; and other monitoring and evaluation documentation) to assess the quality of quantitative and qualitative data and make assessments of project results in relation to the project design and targets set. The evaluator should also review key research and data at the global and national levels relevant to the content of project. All relevant policy and strategy documents at the national level (including National Sustainable Agriculture Development Plan 2010-2030, Five Year Strategic Master Plan, the National Land Policy for Sierra Leone, and others) will be used and referenced as well as research produced by other organisations which is relevant to the project.

Primary Data

In-depth interviews and focus group discussions may be conducted with stakeholders, including project staff, government agency representatives, community leaders, community members and project participants. Concern and the final evaluator together will select communities to visit from a list of targeted communities; focus groups will be conducted separately for men and women, and will include youth.

Primary quantitative data collection is not required but may be proposed by the evaluator to supplement existing project data if deemed necessary to answer the evaluation questions.

8.0 Evaluator Characteristics

The independent evaluator should be a suitably-qualified and experienced consultant or consulting firm. Key team members, their affiliations, and disclosure of conflicts of interest must be listed in the proposal and in an annex to the evaluation report. The consultant will coordinate closely with the Concern team regarding tool finalization, evaluation methodology, timeline, and draft report finalization.

Requirements for Lead Consultant

• Masters but preferably Ph.D in Agricultural Sciences, Food Science/Nutrition, or related discipline
• At least five years’ demonstrated experience leading evaluations or conducting research/evaluations of international development projects, preferably livelihoods projects
• Experience with applied research and translating findings to recommendations and action
• Strong understanding of community, community market systems and the interaction of community stakeholders and the linkages to government and private systems
• Excellent English writing skills (Concern will not be providing editing services), ability to deliver a quality product that meets Concern’s and DFID’s standards on time
• Team player and willingness to lead this evaluation with Concern’s Livelihoods team
• Adaptable, flexible, and able to spend several weeks in rural Sierra Leone leading the evaluation and dissemination events in-country
• Excellent communication and facilitation skills with which to foster effective relationships with staff and project stakeholders in country
• Recent experience with Sierra Leonean agriculture and economic system and policy landscape
• Ability to mobilize and manage enumerators and support staff in-country

Desired but not essential

• Prior major donor evaluation experience, preferably DFID
• Language skills in Krio, Temne, and/or other local languages

9.0 Expected Timeline and Deliverables

The final evaluator will submit the final report to Concern who, in turn, will submit to DFID. The main body of the report must be limited to 50 pages (excluding annexes). The report should include a summary of findings according to the evaluation topics listed above.

9.1 Key Tasks of the Evaluation Team Leader

The following tasks are expected to be completed under the consultancy. All tasks under pre-field work should be completed and approved before commencement of field work period.

Pre-Field Work

• Review project documents and resources to understand the project
• Participate in a call with Concern Sierra Leone to review the scope of work
• Participate in a planning call with Concern to discuss the field evaluation schedule, team composition, and methods
• Produce a brief inception report
• Develop methodology including creation of data collection tools, training and facilitation guides, and sampling plan. Methodology and tools require approval by Concern.
• Recruit enumerators for data collection, as necessary

Field Work

• Train final evaluation team members on objectives and process of the evaluation including evaluation tools
• Lead the team in person to complete the collection, analysis, and synthesis of supplemental information
• Interpret both quantitative and qualitative results and draw conclusions, lessons learned, and recommendations regarding project outcome
• Lead an in-country dissemination meeting with key government, public, and development stakeholders, with a PowerPoint slideshow and oral presentation deliverable, no longer than 60 minutes

Post Field Work

• Draft report on or before November 15, 2019.
• Prepare report and submit to Concern on or before November 30, 2019. The final report will be free of grammar errors and formatted according to DFID guidelines, with input and comments from Concern addressed.

9.2 Proposed Timeline

The length of the consultancy is expected to be approximately 34 working days as illustratively detailed below. Exact working days should be proposed by consultant with the listed deadlines in consideration.

Working Days

Timeframe

Consultant location

Major task

5 days

October

Any location

Review of documents, complete inception report, methodology, and tools

1 day

October

Any location

Coordination with Concern to prepare FE team and schedule

3 days

October 2019

Travel time*

Travel To/From Tonkolili, Sierra Leone

10 days

Sierra Leone

Field work, team data analysis and discussion, preparation of draft report and dissemination of results. All field work will be completed on or before October 31, 2019.

15 days

November 2019

Any location

Preparation of report and annexes. Final report is due on or before November 29, 2019.

*Travel time should be adjusted based on consultant base location.

9.3 Final Deliverables

Over the course of the consultancy period, the following are expected to be completed, submitted, and approved by Concern:

• Inception report including literature review, final methodology, and a detailed plan for fieldwork and post-fieldwork.
• Data collection tools.
• As appropriate, financial invoices and supporting documentation for payment.
• Completion of fieldwork by 31^st October 2019.
• Lead an in-country debriefing for Concern when fieldwork is completed.

At the conclusion of the fieldwork period, the consultants are expected to complete the following deliverables:

• Submit final tools and all raw data collected to Concern.
• Prepare a draft report in line with the guidelines and submit to Concern with at least 5 working days’ time for review.
• Prepare the final report, addressing responses and comments from Concern on the draft, for submission to DFID on or before 29^th November 2019. It is expected the final report will be of exceptional quality, written in British English and free of all formatting errors. The lead evaluator will be responsible for editing in-line with this structure and ensuring the final report is free of all grammar errors.
• As appropriate, standard financial invoices and supporting documentation for payment(s).

10.0 Budget

The total budget for the evaluation will be negotiated with Concern and the lead evaluator in compliance with DFID rules and regulations. In addition to consultancy fees, Concern will cover international and local travel, visa expenses, field lodging, and costs associated with field work to conduct the final evaluation. All anticipated costs should be included and detailed in the budget. Proposals submitted without a detailed budget will not be considered. Insurance and medical expenses, including vaccinations, are not covered by Concern. If other expenses are anticipated, these must be approved by the Country Director in advance.

11.0 PROPOSAL

All interested persons/consultants/consultancy firms should submit an application including the following:

1. CV(s) of the lead person(s)
2. A list of projects, clients, and dates of previous works
3. A sample of a similar evaluation report undertaken by the applicant
4. Technical proposal that includes methodology and a detailed workplan
5. Financial proposal detailing all cost elements with clear budget item lines and unit costs, number of units, and total unit costs. Financial proposal should include any and all and all costs associated with training, enumerator fees, per diems, and accommodations in Tonkolili, transport to and from Magburaka, Value Added Tax (VAT) or Goods & Services Tax (GST) or Sales tax if applicable.[6] Concern will provide transportation to and from field sites from Magburaka.

[1] 2015 Population and Housing Census, Statistics Sierra Leone

[2] Poverty Profile of Sierra Leone 2013; World Bank and Statistics Sierra Leone

[3] Ilesanmi OS, Fawole O, Nguku P, Oladimeji A, Nwenyi O. Evaluation of Ebola virus disease surveillance system in Tonkolili District, Sierra Leone. Pan Afr Med J. 2019;32(Suppl 1):2. Published 2019 Jan 21. doi:10.11604/pamj.supp.2019.32.1.14434. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6445398/

[4] BRACE II Baseline Evaluation, Concern Worldwide, August 2013.

[5] In 2018, some chiefdoms were split or border re-drawn so while this project started in 7 chiefdoms, it is now technically in 10 chiefdoms, however the scope and communities has not changed.

[6] Budget and financial proposal should be inclusive of VAT/GST/Sales Tax, and/or Withholding Tax as applicable. Consultants should notify Concern if they propose/are required to charge taxes. Where so notified Concern will pay such VAT/Sales Tax (if any) as may be chargeable subject on the receipt of an appropriate VAT invoice. Withholding Tax should also be included in the budget where applicable per Section 117 of the Income Tax Act (5% for payments made to contractors who are resident in Sierra Leone and 10% for payments made to contractors who are non-residents). https://www.nra.gov.sl/businesses-and-organisations/withholding-tax

Safeguarding at Concern: Code of Conduct and its Associated Policies Concern has an organisational Code of Conduct (CCoC) with three Associated Policies; the Programme Participant Protection Policy (P4), the Child Safeguarding Policy and the Anti-Trafficking in Persons Policy. These have been developed to ensure the maximum protection of programme participants from exploitation, and to clarify the responsibilities of Concern staff, consultants, visitors to the programme and partner organisation, and the standards of behaviour expected of them. In this context, staff have a responsibility to the organisation to strive for, and maintain, the highest standards in the day-to-day conduct in their workplace in accordance with Concern’s core values and mission. Any candidate offered a job with Concern Worldwide will be expected to sign the Concern Staff Code of Conduct and Associated Policies as an appendix to their contract of employment. By signing the Concern Code of Conduct, candidates acknowledge that they have understood the content of both the Concern Code of Conduct and the Associated Policies and agree to conduct themselves in accordance with the provisions of these policies. Additionally, Concern is committed to the safeguarding and protection of vulnerable adults and children in our work. We will do everything possible to ensure that only those who are suitable to work or volunteer with vulnerable adults and children are recruited by us for such roles. Subsequently, working or volunteering with Concern is subject to a range of vetting checks, including criminal background checking