GIZ Project Evaluation Consultancy 19 views0 applications

Final evaluation Terms of Reference

Installation of Prepaid water metering and hybrid power supply & pumping system project

Project Title

Installation of Prepaid water metering and hybrid power supply and pumping systems in South Sudan

Geographical coverage: global; region; country(ies)-please specify

Greater Equatoria region, South Sudan:

i) Gumbo in Juba and Yei town in Central Equatoria state

ii) Yambio town in Western Equatoria state

iii) Torit town in Eastern Equatoria state

Program/project lifespan (from mm/yy to mm/yy)

1^st November 2020 to 31^st December 2021

Evaluation commissioning manager

Humanitarian Project Manager

Evaluation manager **

Programme quality coordinator

1. *Introduction

South Sudan, the youngest nation in the world, gained its independence in 2011 after a 2010 referendum. But just two years later civil war broke out which affected and continues to impact the life of millions of its citizens and make many of them refugees to neighboring countries, IDPs. The first peace deal signed in 2015 collapsed after just a couple of few months and renewed violent clashes between the warring parties increased across much of the country. On February 2020, a revitalized peace agreement on the resolution of conflict in South Sudan was signed. This latest peace process has promised new hope for the country and there has been an improvement in the general security situation in the country. However, ambushes and attacks along highways, intensification of inter-communal violence (ICV) and revenge killings have continued in some areas of the country and continued to be the causes for the killing of many civilians and humanitarian workers, displacement of people, abduction of women and girls, etc. Human rights, humanitarian situations, food security and economic conditions in the country remain dire, with an enormous impact on civilians.

The cumulative effects of years of conflict and violence has caused significant destruction to various social services including the different WASH infrastructure country wise in general and specifically in the greater Equatoria region – especially Yei, Torit, and Yambio – which continues to impact the lives of local communities.

Yei Town: Arising from the prevailing security situation and associated displacement of impacted communities, Yei town has a population of more than 30,000 people in its municipality. Given the recent peace agreement signed earlier this year, there are indicators that more people are returning to Yei town due to increased availability of services; thus many are abandoning villages for the town in search of medical services, food, security and education for their children. Yei Payam in particular has seen an increase in its population of South Sudanese returnees settling along Kaya Road, Juba Road, Maridi Road and Lasu Road close to the town. Most of the households are vulnerable and require support to access WASH services. Additionally, many residents of Lainya, Morobo, Tore, Mugwo, Otogo, Kupera, Mukaya and Lujulo areas migrated to Yei town for safety. This has affected access to WASH services which remains a challenge due to the upsurge in the population within the town center.

Torit: This is one of the most volatile and conflict prone areas of Eastern Equatoria State. Torit town is an administrative center and the headquarter of Eastern Equatoria State, with an estimated population of 58,864 (29,187 females and 29,497 males). Much of the population depends on business activities as their source of livelihood, and others rely on subsistence agriculture. The town is the state headquarter of various business enterprises, education, health, and religious institutions.

Due to conflict in the state, the WASH infrastructure has been severely damaged or destroyed leaving the entire population to depend on open and unprotected water sources served by River Kineti, as well as a couple of boreholes and seasonal rainwater. Conflict has displaced a number of people from Torit town with many fleeing as IDPs to safer areas of the country, and others as refugees to the neighboring countries.

Yambio: Yambio town is located within the Western Equatoria State and borders the Democratic Republic of the Congo (DRC). According to an IOM Assessment (2018), Yambio town has an estimated population of 113,051 individuals, consisting of 29,447 IDPs, 39,311 returnees and 44,293 host community members. With the formation of the government of national unity, the town has witnessed the return of South Sudanese refugees from the neighboring countries of Uganda, Sudan and DRC. The returnees continue to cause severe strain on the already limited resources that support the town’s population, including core WASH infrastructure. Yambio town was the center of significant armed clashes and widespread displacement in 2016, and prior to this there were persistent conflicts between the South Sudan People’s Defense Forces (SSPDF) and local authorities in Yambio and in neighboring counties, leading many people to seek refuge in the town.**

2. *The Project background

As mentioned above the project was under implementation in Torit town in Eastern equatorial state, Yambio town in Western Equatoria state, Yei town and Juba in Central Equatoria state targeting 150,000 people (78,000 women and 72,000 men) living within the project areas.

The overall objective of the project is to improve the access to safe and clean drinking water to the targeted population in Juba, Yei, Yambio and Torit towns. The key activities of the project are installation of new pre-paid smart water meters/ATMs complete with off-grid solar power supply and power storage, installation of solar PV power supply generator, pump installation and borehole drilling and enhancing the capacity of water utilities in the management of the urban water supply system.

The activities are geared towards the overall goal of supporting social development and economic growth by promoting efficient, equitable and sustainable development and use of available water resources, and effective delivery of water and sanitation services in South Sudan.

All the hardware aspects of the project were directly implemented by Oxfam the whereas three local partners were engaged to implement the software aspects in the three projects. The local partners focused on community engagement activities, awareness raising, training and ensuring sustainability of the project.

3. Objectives of the project Evaluation **

The purpose of this final evaluation is to gage the performance of the project against the predefined key performance indicators and measure progress towards achievement of the overall goal and specific objectives.

Specific objectives:

v To assess the extent to which the project has delivered against its objectives and expected result and activity implementation

v To assess the materialization of the project theory of change in due course of the project implementation.

v To identify adoptability and flexibility in its intervention to the changing humanitarian context.

v Assess the community perception in terms of the utilization of new technologies.

v To draw key lessons from the project and incorporate them in recommendations that will help inform the design and implementation of future similar interventions.

v Document the findings of the project and the learnigs inluding dissemination to the key audience and stakeholders

1. Key questions of the evaluation

The scope for evaluation is determined in line with Oxfam’s Evaluation policy, and relevant evaluation criteria (relevance, effectiveness, efficiency, impact and sustainability; Gender sensitivity. The following are the key questions that need to be addressed and explored.

Key Questions and Evaluation Criteria

a. Appropriateness/Relevance

· To what extent does the program respond to priority issues?

· To what extent are the objectives of the project objectives still valid?

· Do stakeholders care about the project and believe it makes sense?

· How were the different needs of women and men integrated into the design and implementation of the project and how could gender be better mainstreamed?

b. Effectiveness

· What have been the major achievements of the project in relation to the respective objectives?

· What factors have contributed to achieving or not achieving intended project outcomes and set-objectives

· Identify any exceptional experiences/achievements of the projects e.g. case studies, stories, best practices

· Were the projects’ activities implemented in a timely manner as was planned?

c. Efficiency

· Was this project the most cost effective as compared to similar projects?

· Were the resources for running all the activities available, adequate and was this the best use of resources?

· Is the relation between input of resources and results achieved appropriate and justifiable?

· Were there substantial cost over-runs (extension of budget) to complete the project?

d. Accountability

· Was there functional accountability system during the project implementation to share information, ensure participation, regularly collect feedback complaints and feedback form the community and provide on time response?

e. Sustainability

· Will the project contribute to lasting benefits? Which organizations could/will ensure continuity of program activities in the program areas?

· Is there evidence of organizations/partners/communities that have copied, upscaled or replicated project activities beyond the immediate project area? Is such replication likely?

5. Scope of the Evaluation

The project final evaluation will be undertaken in all project implementation locations listed above. The evaluation will focus on the collection of relevant data for a set of indicators outlined in the project proposal. Though the project logic model includes mainly quantitative indicators, qualitative data will be used to get in-depth information about the project’s contribution in the life of the targeted community and acceptance by the community.

The evaluation will also undertake a case study of the innovative technology of prepaid water metering and dispensing system that has been introduced in the targeted locations. In addition, a cost benefit analysis of using solar to power the submersible borehole pumps shall be done comparative to diesel power. The consultancy shall develop a communications strategy and disseminate the findings and all learnings of the project at the end of the evaluation.

6. Evaluation Methodology

In this assessment, the consultant is expected to utilize methodologies well aligned with the tasks at hand. The methodology will involve collection of primary data from the project implementation areas covering a representative sample size of project locations and target beneficiaries. A participatory methodology using HH questionnaire surveys, FGDs, etc. should be used as appropriate for the data collection. Appropriate scientific methodologies should be employed in the sampling techniques and further data analysis. Key data and reports from the online monitoring platforms for the pre-paid water dispensing and metering and the solar water pumping systems shall also be considered during the evaluation.

Stakeholder participation: Design and implement a survey which includes participation of project primary stakeholders who may include the urban water utilities, representatives from the respective local authorities and sector offices from the municipalities, Water Users committees, water technicians, water vendors and other stakeholders. In addition, the methodology should spell out how stakeholders will participate at each level during the final evaluation process.

Techniques: the combination of quantitative and qualitative techniques needs to be considered to this end line. Creativity in use of techniques is at discretion of the consultant. However, all techniques must be globally acceptable randomized techniques.

Gender: The consultant shall explicitly address gender in the final evaluation different level during the survey. Take note that gender inclusion is not limited to only responses in the questionnaires.

Data quality: The consultant to design techniques of data collection which ensure and enforce data quality with quick steps back to revalidate data collected. All data shall meet standards such as accuracy, reliability and validity of data. As part of data collection preparation, the consultant shall conduct training of enumerators within 3 days including pretesting.

Data collection steps should generally include: –

· Development of structured questionnaires and checklists to facilitate data collection in agreement with the project implementing partner

· Identification and training of enumerators and pre-test of questionnaires before actual field exercise

· Develop work plan with a detailed description of the methodology, study tools and instruments for data collection and analysis and the time frame given for the survey.

· Arrange for pre- testing and finalization of the study instruments based on the comments from different stakeholders at Woreda level

· Collect, organize and analyze data obtained from the field and from secondary sources

· Review literature related to the assignment and attach a full bibliography of the references to the final report

7. *Tasks and deliverables

7.1 Key Tasks

· Review Project documents including Project result framework and propose the most appropriate evaluation design for this project and develop end line methodology.

· Map relevant research that has been completed or is planned by partners in each project location pertaining to the thematic programming areas.

· Whenever relevant, disaggregate data by sex (male, female), age (children, adolescent, young people), conflict profile (IDPs, refugees, returnees, host communities), and disability

· Process, analyse and present the data in a user-friendly format

· Presentation of fidnings to keystakholdes in a one day validation workshop that will be organized by Oxfam and to target key stakeholders of the project

· Produce a report that will be widely shared with partners and stakeholders as an important tool for mapping and guiding the implementation of the project.

7.2 Key deliverables

· Development of an inception report highlighting details of the deliverables and methodologies, detailed tools, and instruments (First of all the inception report will be approved by Oxfam before allowing any end line activities. The agreement with the consulting company can be cancelled if the inception report is not approved). This would include all practical operational tools that would be deployed in the survey, the enumerators to be engaged in the survey and the work plan including survey and report deliveries.

· Enumerators Training and Survey Questionnaire Testing – the consultant will be responsible to train the enumerators which is agreed as per inception report, and test the tools agreed. Refinement will be checked after field testing;

· Survey Execution – According to the agreed sample size and area, beneficiary /community and stakeholder coverage. If possible, the data collection should be conducted with the support of GPS to map the respondents for future reference.

· First Draft Evaluation Report – The outline for this delivery could be agreed in the inception report. But it should meet the survey objectives put in the TOR. The timing frame could also be included in this TOR, as well.

7.3 Payment schedules

1. 25% upon submission of an inception report, two weeks (10 working days) after signing of the contract. This report will outline in detail the key scope of the work and detailed study methodology; a work plan/schedule of tasks designating a team member with the lead responsibility for each task and deliverable (output); sources of data; and a data reporting plan.
2. A draft report will be presented to Oxfam by the research team during a stakeholder workshop at a date to be agreed. The evaluation team will facilitate the workshop, which will be aimed at discussing preliminary findings and conclusions of the study.
3. 75% upon submission of a final End line report which should include presentation of the findings and learnings to the key project stakeholders, recommendations and lessons learnt.

8. *Tentative schedule

The study should be completed within a maximum of 30 days from the first day of commencement of the task. The consultant should submit the detailed breakdown of the evaluation timetable based on the following major activities as a guideline or suggestion:

· Development of the evaluation design, finalization of the evaluation matrix, sampling techniques & formula

· Development of research instruments (questionnaires, focus group discussions guides, interviews guides etc.)

· Review of documentation

· Data collection and field visit

· Data analysis and report writing

· Preparation and submission of the first draft report

· Meetings with Oxfam management, program/projects staff and stakeholders on the initial findings and recommendations

· Incorporation of comments and finalization of the evaluation report

9. Consultant team (qualifications and skills needed) **

The consulting firm/team should have experience and conversant in conducting project end line survey, case studies and evaluation for humanitarian response and development programme in general and WASH project/programme specifically, and it should be well acquainted with the current situation in South Sudan. The consulting firm/ evaluation team should come with the following different mix of qualifications and skills:

· Higher university degree in a Water Sanitation and Hygiene (WASH) related field

· Proven experience in conducting quality end line surveys, case studies, evaluations and assessing development and humanitarian response programmes.

· Extensive experience working with international humanitarian organizations globally and specifically in South Sudan.

· Sound technical knowledge extensive experience in WASH projects/programmes

· Excellent attention to detail and evidence of quality outputs from previous assignments.

· Strong analytical and research skills

· Fluent in verbal and spoken English

· Good facilitation skills

1. Roles and Responsibilities

The consultant should:

· Be familiar with the resilience and humanitarian response program (more specifically in WASH related programme) in South Sudan and specifically in the response areas.

· Agree with Oxfam team (management, technical leads and MEAL team) on the methodology of the final evaluation.

· Lead on the evaluation survey process and outputs

· Draft and finalize data collection tools

· Collect and analysis data

· Submit the draft end line survey report to Oxfam

· Submit the final Evaluation report incorporating comments and inputs to Oxfam

· Cover all the costs related to the data collection, analysis and report write up (including costs of transportation)

Oxfam will:

· Facilitate the work as per the specified activity and timeframe in the outline above

· Provide relevant documents to the consultant

· Effect payments according to the agreement

· Provide timely feedback on the evaluation report from all relevant staff members.

11. Budget

Submit a financial proposal that indicates all-inclusive costs for conducting the survey except for commitments listed above.

The consulting firm shall bear all costs associated with the preparation and submission of the proposal. The consultant shall submit ONE (01) copy of technical and ONE (01) copy of financial proposal in electronic form or in a single envelope. The technical and financial proposals should be clearly marked and should include the name and detailed contact of the consultant/ firm.

All costs should be quoted in USD and will remain valid up to sixty days (60) from the day of proposal submission.

12. Confidentiality and data protection **

All the out-puts – Final evaluation report, data base, etc., produced under this assignment will not be disseminated in part or whole without express authority from Oxfam South Sudan. Thus, the consultant firm shall not produce these materials in any form (electronic, hard copies, etc.) to a third party without written permission from Oxfam South Sudan.

13. Research management

The commissioning manager is Humanitarian Project Manager for Oxfam South Sudan Country programme. The consultancy will also be coordinated with the country programme quality coordinator or Oxfam MEAL Team who will constitute part of the review committee which will be responsible for overseeing the work of the consultancy firm and ensuring the effective implementation of the final evaluation. Interaction with the review committee will be at specific points such as the inception, draft and final reports.

14. APPLICATION REQUIREMENTS **

Interested candidates in the position are expected to provide the following documentation:

· A technical proposal with detailed response to the TOR, with specific focus on addressing the scope of work and methodology to be used

· Sample of previous works on project End line surveys, case studies, evaluations, etc.

· Initial work plan based on methodology outlined, and indication of availability

· A financial proposal detailing the daily rate expected, transportation costs, accommodation costs, etc. and initial work plan

· Company profile or CV including a minimum of 3 traceable, recent and relevant references to this task

· Team composition with a lead consultant, and a CV of each person to be involved in the assignment, including relevant experience

Annex 1. Recommended outline of the Final evaluation report

1. Cover page

· Title

· Project title /affiliate identification code

· Date that the report was finalized

· Oxfam logo (unless not appropriate)

2. Table of contents

3. Glossary

4. List of abbreviations.

5. Executive summary that can be used as a stand-alone document

6. Introduction, stating objectives of the end line survey

7. The intervention and context

8. Methodology, including an indication of any perceived limitations of the evaluation**

9. Presentation of the findings and their analysis

10. Conclusions

11. Learning and Recommendations

12. Appendices:

· Terms of reference

· Summary table with list of indicators and evaluation values

· A list of interviewees (name, function and working environment) and places visited.

· List of documents and bibliography used.

· Link to Methodological

ü Evaluation instruments such as questionnaires and interview guides