PROJECT FINAL EXTERNAL EVALUATION of PEAK project 105 views0 applications

PROJECT FINAL EXTERNAL EVALUATION

PEAK Project / Promoting Education For All in Kono –

1. General information

1.1 About Handicap International

Handicap International is an independent and impartial aid organisation. On January 24^th 2018, the global Handicap International network became Humanity & Inclusion. This network is composed of a Federation which implements our programmes in the field in around sixty countries and of eight national associations. These programmes or national associations are known as “Handicap International” or “Humanity & Inclusion”, depending on the country.

The organization is specialized in the field of disability, works with partners to ensure that Persons in disabling situation may recover their capacity to act, by improving living conditions and by greater social participation. The organization seeks to increase the capacity of populations to satisfy their own needs and exercise their fundamental Human Rights. Handicap International uses an inclusive approach to programming by ensuring that Persons with Disabilities have access to those services intended for everybody in society.

1.2. About Handicap International in Sierra Leone

Handicap International (HI) has been working in Sierra Leone since 1996 and has implemented several programmes including Social Inclusion and Rights, Inclusive Livelihoods, Physical Rehabilitation, Mother and child Health, Ebola health emergency, Pain Management, Mudslide Emergency Response and Inclusive Education project.

Currently, HI is implementing three different projects for the promotion of quality education:

· Girls’ Education Challenge Transition (GEC-T) from April 2017 to July 2021;

· Educate a Child (EAC), from September 2016 to August 2019

· Promoting Education for All in Kono (PEAK) “from 30/10/2016 to 31/10/2019.

HI also currently implements other unique projects including Health – Touching Mind Raising Dignity (TMRD) on Mental Health and Accessibility & Inclusion Technical project through providing technical support to partners, agencies and Government Ministries of Health, Education and Social Welfare Gender and Children’s Affairs.

2. Evaluation context

2.1 Presentation of the project to be evaluated

Project title :Promoting Education For All in Kono – PEAK

Implementation date :Nov. 2016 – Oct. 2019-Three (3) years duration.

Location/Intervention zones : All 14 Chiefdoms of Kono District – Eastern Province- Sierra Leone

Co-applicants : Community Action for Rural Development (CARD), Advocacy Network for Empowerment & Good Governance (ANfEGG) and Youth Dream Center (YDC).

Target groups : Vulnerable children including children with disabilities, Ebola affected, orphans, single parent children and out of school children.

The project also targets local school governance structures (SMC, CTA, MC) of 70 schools (14 pilot and 56 satellite), 238 teachers, 12 Ministry of basic and senior secondary education school supervisors, community and district stakeholders, 5 civil Society organizations – sub grantees and school community members benefiting from adult literacy learning and the community awareness on inclusion and inclusive education.

Project budget : 1,200,000 Euro (75% donor and 25% matched-fund)

Project objectives

Overall Objective: Impact – To contribute to a universal and sustainable strategy for the access to quality education for all children in Sierra Leone.

Specific Objective(s) – To promote primary school structures, local actors and communities’ ownership in implementing an inclusive education model to all children including vulnerable girls, children with disabilities and special education needs, out of school children and Ebola affected children in Kono district.

Expected results and indicators

1. Quality education for all children is promoted through the implementation/creation of Child-Friendly School (CFS) models in 14 pilot and 56 satellite primary schools in Kono district.
2. Local micro-project initiatives increase capacity and ownership by Civil Society Organizations (CSO) and community stakeholder structures to maintain a sustainable implementation of quality inclusive education at district level.
3. Kono communities and parents are promoting and working towards education for all children and the reduction of stigma against the most vulnerable children.

Main activities implemented

R1

· Adapted 14 schools into a Pilot Model accessible and inclusive school, one in each chiefdom.

· Provided assorted teaching and learning materials and equipment for the 14 model schools.

· Developed and Produced one In-service inclusive education teacher training curriculum.

· Conducted in-service teacher training for 238 teachers and 11 school supervisors.

· 280 children with disabilities have received technical medical assessment, treatment and assistive devices

R2

· Selected, trained and provided small grants to five (5) emerging/grass root Civil Society Organizations (CSOs) in Kono district who implemented micro-project inclusive education initiatives.

· Trained key local education governance structures – School Management Committees (SMCs), Community Teachers’ Associations (CTAs) & Mothers’ Clubs.

· Held regular Education coordination meetings at chiefdom and district level

· Created education platforms and forum at district and chiefdom level and conveyed strategic dialogue meetings

R3

· Completed community awareness raising sessions to community members including parents on stigma reduction.

· Created and supported the functions of 14 children’s clubs

· Carried out community and district level Radio campaigns – panel discussion programmes and airing of jingles.

· Supported 120 members of five (5) adult literacy groups in five chiefdoms through literacy sessions and group initiatives

· Identified vulnerable children, their needs and service providers and create referral pathways.

A Logical Framework has been developed and updated (see annexe 1)

2.2 Reasons for the evaluation

The final external evaluation is required for the following reasons:

• The final evaluation is part of contractual agreement between HI and the donor.
• The final evaluation is part of Monitoring Evaluation Accountability Learning Policy of HI for any project of 2 years or over
• The final evaluation will provide an opportunity to identify key areas and factors for success in this kind of projects and reflect towards improvement of the action in the future

The final external evaluation will show case how HI and the PEAK project specifically have been transparent, accountable in view of respecting all guiding institutional and donor policies to project stakeholders including beneficiaries especially children. The evaluation will inform HI of the effects of this project so that the relevant lessons can be learned from this experience and shared with other HI programmes and services.

Finally, the evaluation will cover all of the project components, both at the local and national level. Based on the realised activities, the evaluation will measure the extent to which the specific and overall objectives, as they were initially defined, have been met. It will identify any gaps in the activities implemented, their causes and their effects on the project; and the contributing factors to success.

3. Evaluation objectives

3.1 Overall objectives and expectations for the evaluation

The final and ex-post evaluation will provide HI, the co-applicant and EU with sufficient information to:

• assess and analyse the project results, efficiency and relevance of the implementation strategy
• review, analyse, provide evidence and document the impact of the intervention through its progress, challenges, barriers and lessons learned with recommendations for a logical and potential next phase.

3.2 Specific objectives

The final external evaluation will target the project “Promoting Education For All in Kono – PEAK” implemented by Handicap International from 1^th November 2016 and 31st October 2019 and funded by EU.

Based on the self-evaluation made with the project team and the co-applicants, HI has defined the below evaluation criteria:

• Changes: to evaluate positive short, medium and long term changes for the target population. The evaluation will focus on 3 sub-criteria: effects, impacts and mitigation.
• Sustainability: to evaluate the positive effects that will be ongoing once the intervention is over. The evaluation will focus on 3 sub-criteria: anticipation, continuity and resilience.
• Efficiency: to evaluate the optimisation of resources (human, financial, logistics, technical…). The evaluation will focus on 2 sub-criteria: flexibility and optimisation.
• Cooperation: to evaluate the participation and implication of the project’s partners. The evaluation will focus on 2 sub-criteria: involvement and results.

The evaluation results are expected to be used by different stakeholders involved in the project according to the different components of the project. HI and the EU will potentially use recommendations related to any sectors. Main aspects expected to be considered and potentially used by national and local stakeholders are:

• Lesson learnt: the main recommendations and evidence will be used to develop new projects and improve the inclusive education project currently run. The users of these recommendations will include HI and the 3 co-applicants.
• Capacity Building: PEAK project has a strong capacity building component focused on 3 co-applicants and 5 sub-grantees. Recommendations could provide useful information in term of building the capacity of local organisation to be autonomous and able to lead a project funded by EU.
• Sustainability of CBRV and model school: Recommendations could provide useful information in relation to sustainability of the CBRV program and model schools in Sierra Leone, the possible development of CBRV policy and strategy and criteria of accessibility, the implementation strategy of CBRV in excluded areas, the role of civil society organisations and local authorities in the CBRV and in accessibility works. The users of these recommendations will include HI, the Ministry of Social Welfare, Gender and Children’s affairs, Ministry of Works and Public Assets, the EU and others stakeholders involved in disability issues.
• The assistive devices process: HI in Sierra Leone is providing assistive devices in 1 other project and brings a technical support to others partners. The main recommendations will be used to improve the process of provision of assistive devices and make it more sustainable. The users of these recommendations will include HI and the service providers.
• The inclusive education: HI is involved in the design of the national Inclusive Education Policy and the Inclusive Education Curriculum for teachers training in in-service will be useful for the implementation of the new policy. The main recommendations could provide useful information in relation to sustainability and dissemination of inclusive education curriculum.

3.2 Evaluation criteria and questions

• Changes
• Relevance
• Efficiency
• Cooperation
• Sustainability

KEY SUGGESTIVE QUESTIONS

Changes

• Has the project produced any significant positive changes in the life of the beneficiaries and are these changes sustainable?
• Did the project sufficiently take into account and/or avoid generating negative effects (environment, economy, security, etc.)?

Relevance

• To what extent has the project met the needs of its direct beneficiaries?
• Are all the project’s activities a clear part of HI’s mandate?
• Has the project sufficiently adapted its actions to the context of the country?

Efficiency

• Did the intervention method used enable results to be achieved in a cost-efficient manner?
• To what extent did the project team optimise resources?

Cooperation

• Did partners contribute sufficiently and optimally to the outcomes of the project?
• Were partners kept sufficiently up-to-date with the progress being made on the project?
• Did they take part in the different phases of the project cycle?

Sustainability

• Are the actions implemented during this project effectively sustainable?
• Has the project invested the means necessary for ensuring the continuity of activities after the project’s closure?
• Has the project significantly reduced the vulnerability of all the beneficiaries?

4. Evaluation methodology

4.1 Collection method

Possible methods for data collection would include:

• · Internal documents review: Intern-Service Project reviews report, ROM evaluation, interim reports, self-evaluation, exit strategy, monitoring tools.
• · External document review such as inclusive education policy, Inclusive education curriculum, project reports implemented in Sierra Leone, Inclusive education project report implemented in similar context, research on education and disability.
• · Interview with the co-applicant and the 5 CSOs.
• · Interview with stakeholders and final beneficiaries
• · Focus group with parents from literacy groups, CBRV, children from children’s club, teachers in models school, community members
• · Survey by questionnaire for teachers and vulnerable children attending the model schools, satellite schools and schools not targeted by the project.
• · Collection of story telling
• · Field visits
• · Data analyse tools

Participatory approach during data collection is expected by not only considering stakeholders including the co-applicants as providers of data only but also involving them in initial analysis; for this, workshops could be organised at the beginning and at the end of the evaluation.

4.2 Actors involved in the evaluation

With respect to the time and resources available, and in relation to the methodology proposed, evaluation should rely on a participatory approach to increase ownership, empowerment and usability of the results.

The reference group for the evaluation is made of:

• The Country Director
• The Operational Manager and the Deputy Operational Coordinator for Sierra Leone Program
• The Inclusion Specialist for Sierra Leone program
• The Project Manager

A list of the key stakeholders involved in the intervention and to be consulted during the evaluation is in annex 2.

A team coordinator will be clearly appointed. The team coordinator will be ultimately responsible for the whole work and all deliveries; s/he will also act as the contact person with the evaluation managing unit.

5. Principles and values

The ethical principles adopted by HI can be transposed into 8 interdependent recommendations specific to the data management:

• Guarantee the security of all the partners involved;
• Ensure a person/Community-centered approach;
• Obtain the subjects’ free and informed consent;
• Ensure referral mechanisms are in place;
• Ensure the security of personal and sensitive data at all stages of the process;
• Plan and guarantee the use and sharing information;
• Ensure the expertise of the teams;
• Respect the HI policies and especially the Child Protection policy,the Protection of beneficiaries from Sexual Exploitation, Abuse and Harassment (PSEAH) policy and the Anti-fraud, bribery, and corruption policy. A code of conduct will be signed by the consultant who will also receive a sensitization on these 3 policies.

The evaluation report remains the intellectual property of HI and EU.

6. Expected deliverables and proposed schedule

6.1 Deliverables

· The facilitation and documentation of a field working session including a workshop with partners (Co – applicants)
· 1 inception report
· 1 draft report in English for review and comments by project team (responses coordinated by a team coordinator), Operations Manager, Country Director & desk.
· 1 second draft report in English incorporating comments
· Final report in English of 30 pages maximum (25 pages of report and 5 pages summary (without annexes)
· 1 oral presentation + power point format
· Written reports will include executive summary report/brief, which can be shared with all partners and stakeholders of the project
The final report should not exceed 30 pages (without annexes). The report will describe the evaluation, the evaluation findings, lessons learnt and recommendations. It should include:

• Executive summary of maximum 5 pages
• Introduction
• Description of the intervention evaluated
• Analysis of the information compiled
• Results
• Conclusions
• Lessons learnt
• Recommendations

· Annexes:

• Initial and final terms of reference of the evaluation
• List of persons interviewed
• List of documents reviewed
• Final timetable of evaluation
• Tools used for data collection
• Preliminary results presentation and report of the meeting
• Detailed analysis if relevant
• Financial Report

6.2 Evaluation date and schedule

The work plan below is estimated by the team as a basis and should be revised and precised by the consultant according to the technical offer made.

The proposed steps for the evaluation are:

• Inception phase desk-based review (in Lyon) and initial gathering of information
• Core data collection, done in Kono and centrally with the co-applicant and key stakeholders, based on a previously designed and agreed methodology
• Analysis and presentation of first findings
• Reporting

A more detailed example of a work plan is presented in Annex 3.

Schedule

The evaluation is expected to take place in October 2019 (last month of the project) with around 14 days Sierra Leone.

Draft report is to be submitted to the reference group by 30th October 2019.

Final report of the external evaluation will be due no later than 15th November 2019.

7. Resources

7.1 Expertise required from the consultant(s)

• Qualified professional in Measurement and Evaluation
• Qualified with at least 5 years’ experience in project planning, designing, management, monitoring and evaluation.
• Experience in working in sub-Saharan regions and its remote communities including Sierra Leone.
• Proven experience of evaluation and learning of education projects. Experience in inclusive education will be an asset.
• Excellent reporting
• Proven experience of designing and implementing relevant evaluation tools including KII (Key Informant Interview), FGD, KAP

7.2 Budget allocated to the evaluation

The maximum available budget is 13,000 Euros.

Payment will be made in three instalments: 20% at contract signature, 40% at the finalization of the field work, and 40% once the final evaluation report has been validated, provided the schedule and deadlines defined together and the quality criteria referred to in the contract have been met.

The budget should be detailed – including calculations – and include: Fees and per diem
Medical and Repatriation insurance (proven at contract signature)
Third party liability Insurance (if/when required)
Travel expenses to Sierra Leone – including visa expenses
Accommodation within Sierra Leone – both in Freetown and in the provinces (estimated)
Communication costs (internet connection outside HI office, mobile credit)
Workshop or meeting organization and logistics (estimated)
All Taxes (if/when applicable)

Any other relevant cost related to and required for the proper conducting of the evaluation as per all the objectives and requirements detailed in the present ToRs.

I will ensure:

Local transportation from HI office in Freetown to HI Koidu office and the 6 selected communities and stakeholders in Kono for the field visits.

Local transportation to the national partners and donor

7.3 Resources available to the evaluation team (data, document, etc.)

The evaluator will propose himself/herself and/or a team capable of handling the specificity of the thematic area concerned.

Handicap International Federation – Program in Sierra Leone, will provide human resource assistance to support the following activities:

Provide initial contact / introduction to key stakeholders identified to be involved in the process

Support the arrangement of transportation to rural evaluation sites

Support the consultant in the arrangement of focus groups/workshops and translation for local dialects

.8. Annexes / Available under request to [email protected]

Criteria for assessing the quality of an evaluation (see 10. Final report and 11. Assessing the quality of an evaluation) should be included in the annexes to the ToR to enable the evaluator to take these criteria into account when drafting the final report.

ANNEX 1 : logical framework

ANNEX 2: List of the key stakeholders involved in the intervention

ANNEX 3: Proposed steps and an estimated timeframe for the evaluation (indicative)

Inception phase (5 days)

· In the inception stage, the relevant programming documents should be reviewed, as well as documents shaping the wider strategy/policy framework. The evaluation team will analyse the logical framework.

· Interview with key informants, among which could be:

• HI project staff and direction
• Donor: EU
• Main authorities at national and district level
• 3-co applicants and 5 CSO
• Njala University
• 2 service providers and other INGO working in Kono district
• Pre analysis, production updated plan and preparation of field work

On the basis of the information collected the evaluation team should:

• Describe the project context.
• Comment on the logical framework.
• Comment on the issues / evaluation questions suggested or, when relevant, propose an alternative or complementary set of evaluation questions justifying their relevance. Develop the evaluation into sub-questions identify provisional indicators and their verification means, and describe the analysis strategy.
• Present an indicative methodology to the overall assessment of the project/programme.
• Identify and present the list of tools to be applied in the Field Phase
• Propose the work plan for the finalisation of the first phase.
• Confirm the final time schedule.

An inception report shall be prepared.

• Data collection: (7 days)
• in depth document review
• Individual interview with members of committees, key trainers, final beneficiaries
• Focus group with parents from literacy groups, CBRV, children from children’s club, teachers in models school, community members
• Survey by questionnaire for teacher and vulnerable children

· Analysis and presentation of first findings (4 days)

• Analysis of findings: 3 days
• Workshop presentation of preliminary results : 1 day

· Reporting (4 days)

• Production and review by the reference group of 1st draft evaluation report: 3 days
• Production of final evaluation report: 1 day