NIGERIA INCLUSIVE RECOVERY RESPONSE NORTH EAST-END TERM EVALUATION 237 views2 applications

1. Background of Project

Christoffel Blinden mission (CBM), is an international Christian development organization, committed to improving the quality of life of people with disabilities in the poorest communities of the world. Based on its Christian values and over 100 years of professional expertise, CBM addresses poverty as a cause and a consequence of disability and works in partnership to create an inclusive society for all. CBM’s work is founded on the core values of Christianity, internationalism, professionalism, stewardship, honest communication and inclusion

The Boko Haram crisis has tormented the north-eastern part of Nigeria since it began in 2011 leading to the loss of lives and properties worth billions of naira. About 8.5 million people are estimated to be in need of urgent assistance particularly as the activities of the insurgents crippled economic activities and cause severe food crisis, as persons with disabilities are worst hit by the insurgency because of their vulnerability to abuse and limited or no access to humanitarian assistance.

CBM Initiatives cut across Mental Health Services, Eye Health, NTD’s, CBID and WASH Services through the provision of portable drinking water, construction of accessible latrines, waste management and hygiene awareness campaigns with the aim to contribute to the improvement of the quality of life of persons with disabilities affected by the Boko Haram insurgency in the North East of Nigeria.

In November 2016, CBM began the implementation of the acute response phase targeted at people living with disability, elderly, female-headed households, orphans/child headed households and single pregnant women and the project was implemented through four (4) of its partners namely COCIN, GAMMUN, WINN & TAIMAKO in Borno and Yobe States respectively.

Name of Partner

LGA

State

Name of communities

COCIN

MMC

Borno

Bale Galtimari, Molai Kura Primary Schools, Stadium IDP Camp, Kangadari, Ali Waje, Molai Shuwari

GAMMUN

Konduga

Borno

Jakana, Jimtilo, Auno, Kondori

WINN

Konduga

Borno

Kafan Ruwa, Wulari, Sabon Gari, Mandarari, Alau, Mashimari

TAIMAKO

Yobe

Kukareta, Katarko, Gujiba Gari, Buni Yadi I

The NIRR project which commenced in January 2018 to December 2019, secured a no cost extension from January 2020 to June 2020 to provide food assistance through its partners to 1700 households in Borno and Yobe states, reaching over 11,900 individuals.

The overall goal is to contribute to the improvement of the quality of life of persons with disabilities affected by the Boko Haram insurgency in the North East of Nigeria.

Result 1: Most vulnerable households benefit from food distribution (dignity packages).

Indicator: At least 1200 targeted households benefited from the food distribution

Result 2: Increased food production by persons affected by the Boko Haram insurgency in targeted states and communities.

Indicator 1: At least 12 farmer’s groups are supported with farm inputs.

Indicator 2: 180 farmers capacity are built on better farming methods for improved yield.

Indicator 3: At least 70% of the beneficiaries have improved food security and are able to maintain it.

Indicator 4: Farmers groups are transitioned into VSLA groups.

Result 3: Households of persons affected by the Boko Haram insurgency in the target communities are economically strengthened.

Indicator 1: At least 12 self-help groups supported and trained on alternative livelihood options.

Indicator 2: At least 180 people benefit from support services including trading, processing, manufacturing, servicing.

Indicator 3: At least 40% of the micro-enterprises self-help groups are transitioned into VSLA groups.

Result 4: Mainstreaming of disability-inclusive practices into the programmatic work of INGOs and UN agencies working in Northeast-Nigeria

Indicator 1: At least one training/sensitization conducted for other humanitarian actors.

Indicator 2: At least 5 humanitarian partners have mainstreamed disability inclusive practices in their respective programme approach.

2. Evaluation Purpose, Scope and Intended Use

The purpose of the evaluation is to assess the overall project impact during the implementation period as far as possible considering attribution and contribution. It will document good practice and lessons learnt that will be used to improve and inform future programmatic orientation/direction, as well as for external communication and promotion.

The scope of the evaluation will cover activities from January 2018 to June 2020 implemented by all 4 partners.

This evaluation is being carried out as part of the programme development requirements. Its outputs will be used by a variety of stakeholders for different purposes:

v Implementing partners will use the information from this evaluation

ü to understand the impact of the programme on the service lessons users

ü to plan for the sustainability of the programme

ü learned will be used to plan future programming

v CBM will use the information from this evaluation

ü to inform the way in which CBM can partner and work with government to establish accessible, quality livelihood services for persons in emergency settings.

ü to identify areas that may require further strengthening to ensure sustainability and quality of services.

ü to guide provision of technical support to future projects.

ü to feed into practice of CBM and partners.

ü to improve the overall quality and impact of CBM’s work.

Anything on government as an audience? Other INGOs that received mainstreaming training?**

3.Evaluation Objectives

· To assess and document the achievements and challenges of the project objectives thus far, reasons for (non-/under-) achievement and the contributing factors

· To identify other intended/unintended and positive as well as negative changes as a result of the intervention

· To document lessons learnt from project planning, implementation, monitoring and learning that would guide future project management.

4.Evaluation Questions

Relevance:

1. To what extent did the objectives and implemented activities meet the needs and priorities of the beneficiaries?
2. Were the strategies used the most suitable for achieving intended outcomes?
3. Did the project effectively reach the most vulnerable households with focus on person with disabilities?

Coherence:

1. To what extent were contextual factors such as political stability / instability and population movement taken into consideration in the design and delivery of the intervention?
2. To what extent was the intervention coherent with key policies and programmes of other implementing partners in the North East?
3. To what extent was the intervention design and delivery overall in line with humanitarian principles?

Effectiveness:

1. To what extent were the expected outputs and outcomes achieved?
2. Were activities implemented as planned? If not, reasons for it.
3. Was the project sufficiently adaptable to a fluid and insecure context to deliver outputs in a timely fashion and sufficiently achieve targets?
4. Were the monitoring mechanisms effective in providing timely data to inform programming decisions?
5. What factors contributed whether or not activities led to intended outputs and outcomes?
6. To assess the results of the livelihood development activities and their medium to long-term continuation

Efficiency

1. Are overall costs and resources used for the delivery of the intervention justified or would a different methodology of project delivery have achieved more and better results? Was adequate human, financial and logistical resources applied to delivering project outcomes?
2. Could the project outcomes have been achieved at a lower cost?
3. Were outputs delivered in a timely fashion?

Impact: what is the project impact to the local economy and the household food security?

1. To assess the change in the economic/livelihood of targeted households as a result of programme intervention.
2. To assess the change in the economic situation of the individuals and households addressed by the intervention
3. What changes took place as a result of the project implementation whether positive or negative?
4. Were the project’s intended goals achieved?
5. Were any unintended change (s) achieved (positive or negative)?
6. What factors affected the achievement of intended outcomes or otherwise?

Coverage/Disability Inclusion:

1. How have target groups been identified and was this appropriate in the fragile and changing environment?
2. To what extent were persons with disability actively engaged in the entire project process and management?
3. To assess the attitude towards disability and disability inclusion within the communities and the local administration/authorities
4. How did their involvement or non-involvement affect the achieved outcomes?
5. To what extent were the services accessible to the beneficiaries?
6. To what extent were persons with disabilities empowered and benefitted from the project.

Sustainability:

Includes an examination of the financial, economic, social, environmental, and institutional capacities of the systems needed to sustain net benefits over time. Involves analyses of resilience, risks and potential trade-offs. Depending on the timing of the evaluation, this may involve analysing the actual flow of net benefits or estimating the likelihood of net benefits continuing over the medium and long-term.

1. To what extent is it possible to scale up the project beyond the project areas to other communities and states.
2. Identify and propose procedures to ensure that the benefits of disability inclusive awareness and practices continues in the long term.

Gender and Safeguarding

1. The study shall consider gender aspects of the project and report on the equal participation of women and men, of boys and girls in the projects, how their specific needs have been addressed and how they benefitted from the intervention. Data collected should be disaggregated according to sex, age, and disability.
2. The study shall assess what type of safeguarding mechanisms have been put in place and how they were practically implemented – this concerns children, children with disabilities and adults at risk/with disabilities. The report will show how and whether children/adults at risk in the target area were protected from abuse and maltreatment and prevented from practices harmful to their health and development in the course of project implementation.

5.Methodology

The evaluation process will take a participatory approach by involvement of programme stakeholders, especially the beneficiaries (primary, secondary and tertiary). Programme staff and partners will play a strong role in the evaluation.

The evaluator will develop the evaluation methodology in the framework of the available evaluation budget in collaboration with CBM and the programme partner’s representative. He/She is expected to submit a brief inception report where an evaluation methodology should be proposed. The evaluation must meet the principles of being inclusive, participative and interactive, involving both male and female beneficiaries. As a minimum, the evaluation process should include the following key steps:

1. Review of relevant project documents such as approved project description and log frame, PPA,
2. Review project quarterly reports (i.e. post distribution monitoring reports, farmers group monitoring reports, food distribution reports, etc.) and findings from beneficiary feedback mechanisms
3. Review and use of the available routine and evaluation data.

Stakeholder consultation, including partner project staff and management, beneficiaries, community volunteers; key community leaders where applicable, representatives of key Stakeholders and CBM Country Office staff.

The consultant shall develop the methodology (incl quantitative and qualitative methods). Data collection tools and detailed methods as well as roles of the evaluation team members shall be consulted and agreed with the project partners and CBM.

Do-No-Harm, Safeguarding and Data Security:

Regarding confidentiality/ data protection, the evaluator must take all reasonable steps to ensure that the respondent is not adversely affected by taking part in the evaluation. He/she must ensure that informed consent it given according to CBM requirements for all interviews; must keep responses confidential at all times and must not do anything with interview responses that they are not informed about at the time. Also, particular care must be taken with children and teenagers. Permission must be granted from a parent or an officially designated caregiver for interviews with children aged under 18 and CBM’s child safeguarding policy should be applied in all circumstances.

Following data collection and analysis, the consultant will share preliminary findings with CBM and project partners. This shall be achieved through debrief sessions at CBM and the evaluation sites. The sharing of preliminary findings is an opportunity for the stakeholders to hear what the evaluation has found and to be involved in thinking through recommendations. It should include constructive discussions around the key issues identified by the evaluation.

6. Required Expertise of the Evaluator

The evaluation will be conducted by an independent consultant who will work in close collaboration with CBM Nigeria Country Office and the intended partner organisations.

The consultant must have multi-disciplinary expertise with a key background and expertise in public health, livelihood and disability inclusion in humanitarian response.

Persons with disability must be involved in conducting the evaluation study to best assess the inclusiveness of the proposed project components. CBM and the partner organisation can support this.

The evaluator will be selected based on the following criteria:

ü Seven (7) to 10 years’ proven experience in programme design, implementation and evaluation with five (5) of those years in evaluating programmes of similar size and topic.

ü Advanced degree/MSc degree/ in public health, humanitarian response or other relevant fields.

ü Strong background in humanitarian response in post-conflict environments as well as in inclusive development;

ü Knowledge and practical experience in working with health and livelihood programmes;

ü Proven senior-level evaluation experience (at least 5 years) including knowledge of evaluation methodology;

ü Ability to draw practical conclusions and to prepare well‐written reports in a timely manner and availability during the proposed period.

ü Ability to provide strategic recommendations to key stakeholders;

ü Excellent interpersonal and communication skills including ability to facilitate and work in a multidisciplinary team;

ü Analytical skills proven through submission of a past evaluation report

ü Knowledge of disability inclusive practices in evaluations is an added advantage

ü References to include clients and other evaluators

ü Language skills

MANAGEMENT RESPONSIBILITIES

7. commissioning responsibility

CBM is responsible for commissioning the evaluation and provide guidance and institutional support to the external consultant.

CBM Nigeria CO will be responsible for planning and managing the evaluation and checking that quality standards are met, ensuring the evaluation conclusions and recommendations are communicated effectively.

The evaluation team will be identified and approved by CBM jointly with the partner organisations.

CBM CO Nigeria is responsible for the contract and payments.

A joint approval of the final report will be done by CBM and the partner organisations. The final instalment of consultant’s fees will be disbursed following sign off of the final report by CBM.

8. Logistics and coordination

COORDINATION AND LOGISTICS

Country office has responsibility for:

· Overall coordination of evaluation process.

· Recruitment of evaluator.

· Work with partners to agree on TOR, consultants and evaluation process and schedule.

· Gathering documents and data for evaluator in collaboration with partners.

· Organising post-evaluation debriefing with the evaluator, including partners in the process.

· Organising communication of results of the evaluation.

· Support partner in developing a management response/action plan based on evaluation recommendations.

Partners have responsibility for:

· Working with the Country Office to organise meeting schedule for evaluation team.

· Identifying “neutral” and accessible locations for interviews/ meetings to take place (where people will feel free to speak as openly as possible).

· Organising for interviews with beneficiaries and community leaders according to the evaluator’s requests/methodology.

· Organising for interviews with the stakeholders according to the evaluator’s requests/methodology.

· Provision of guidance on security and safety at the evaluation sites**

Evaluators have responsibility for:

· Professional and ethical conduct of the entire evaluation following the do-no-harm principles.

· Evaluation using the OECD DAC evaluation standards and criteria.

· Ensuring participatory and inclusive methodology

· Delivery of the following products (according to agreed schedule):

(i) An Inception report.

(ii) A final evaluation report using CBM evaluation report template.

(iii) Data sets for all collected data (quantitative and qualitative).

(iv) All data collection tools that have been used together with a summarized analysis of the data.

(v) Separate list of persons interviewed, or group discussions conducted, ensuring that personal data is processed in a safe manner

(vi) Actual evaluation schedule/agenda indicating actual working time covered

(vii) A PowerPoint presentation or similar, summarizing the key finding from the evaluation for presentation in a final workshop with partners and CBM.

(viii) Depending on defined needs for target communities, an accessible and easy to understand presentation of the findings and recommendations for the stakeholders.

INCEPTION REPORT

An inception report is expected from the consultant after the literature review and during an inception briefing. The purpose of this report is to ensure that the evaluator covers the most crucial elements of the exercise including the appropriateness and robust methodology to be employed. The inception report provides CBM and the evaluation team with an opportunity to verify that they share the same understanding about the evaluation and clarify any misunderstanding at the outset. The report should reflect the evaluation team’s review of literature and the gaps that the field work will fill. It will also provide a detailed schedule for the evaluation exercise on location.

The inception report shall follow the suggestions of the CBM Inception Report Template and can be amended as needed.

EVALUATION REPORT

A draft evaluation report must be submitted to CBM Country Office at the agreed date. The draft report will be circulated by CBM Country office to key stakeholders (Project partners) for review and feedback. Feedback shall be provided within 2 weeks of receipt of the draft report.

The report should be prepared using the CBM evaluation report template in Appendix 1. It shall comply as much as possible with accessibility criteria – CBM can provide guidance on that.

The final report of the evaluation must be submitted to CBM after review and incorporation of the various comments made at the agreed date. The main aim of the report is not only to feedback on this program, but to improve on the quality of work by the partner and CBM.

The evaluation report is an exclusive property of CBM and should not be released without prior authorization to any other party. The final report will be available through CBM as well as being specifically circulated (by CBM) to the project stakeholders, including the project partners who will be able to use the report freely.

DATA SETS

After final approval and payment of the evaluation, the consultant has to ensure that not personal data from the interviews etc. is available to a third party. This might include the handing over of such data to the CBM CO for storage. Electronic data have to be deleted entirely from the computers of all consultants or other externals involved in this evaluation.

11. Duration and Phasing

The duration of the evaluation exercise shall be 22 working days from a mutually agreed date no later than 8 June 2020. The evaluation will follow the key phases:

Phase I – Desk study: Review of documentation, elaboration of inception report, and development of evaluation tools [2 +1 days]

The consultant will review relevant documentation. Based on this review, he/she will produce an inception report which will include an evaluation plan, methodology and sampling strategy of the data collection for evaluation study. An inception briefing with CBM CO and partner representatives will be held.

Phase II: Field Data Collection (12 days)

This phase of the evaluation will seek to collect primary data on the key evaluation questions explained under evaluation criteria. The evaluation team will use the agreed plan, methodology and sampling strategy to conduct the field work.

Phase III – Data analysis and production of evaluation report [3 days]

The evaluation team will draw out key issues in relation to the evaluation questions and produce a comprehensive report.

The table below summarizes the key activities outlined above

Phase

Activity

Location

Expected Dates

No of Days

Desk study: **

Desk research /literature review

home based

2 days

Submission of Inception Report and evaluation tools

CBM Country Office

1 day

Field Data Collection

Field Visits & data-collection (with debriefing at the end of each visit)

Project sites

10 days

Data analysis and preparation of draft report

Home based

**

5 days

Submission of draft report

CBM Country Office (via e-mail)

1 day

Review of draft report after feedback from CBM and partners

home based

1 day

Submission of final report**

CBM Country Office

1 day

TOTAL

22 days

12. COSTS AND PAYMENTS

12.2 SCHEDULE OF PAYMENT

· Signing Contract: First advance of 10%

· Submission of inception report: Second advance of 30%

· Approval of all deliverables as stated above to CBM, Final payment following receipt of invoice from consultant (60%).

12.3 MODE OF PAYMENT

Bank transfers