Consultant(s) – End Evaluation of South Sudan Red Cross’ CEA project 90 views0 applications

1. Summary

1.1 Purpose: To assess the progress made by the CEA project towards achieving its objectives, unintended results, lessons learned and recommendations for continued implementation.

1.2 Audience: Program and project teams, senior management, branches, volunteers, PNS and Movement Partners, communities and donors.

1.3 Commissioners: Swedish Red Cross (SRC)

1.4 Reports to: The SRC Country Representative

1.5 Duration: 30 days in February and March 2020. A final report to be submitted by 25 March 2020.
Location: National Headquarters of SSRC, branches, and communities in South Sudan
**

1. Background**

Swedish Red Cross and South Sudan Red Cross agreed to implement the project, Community Engagement and Accountability (CEA) which would strengthen the community resilience through the local Red Cross Branches. To build resilience and ownership of projects in communities, the capacities of the local Red Cross branches as well as their accountability towards communities needed to be improved.

The local branches of SSRC are often the first responders to local emergencies and, therefore, also ‘experts’ in terms of what is needed to build the resilience of their communities. The CEA project was started in 2017 and mainly focused on capacity building. The project end-date is 31 March 2020.

The original project had four project locations: Raja; Wau; Terekeka, and Torit. However, in 2019 three other branches were added: Juba, Bor, and Rumbek.

The project uses the RCRC Movement CEA minimum standards as well as the results of the BOCA (Branch Organizational Capacity Assessment) as tools. Gender and Diversity, as well as Safer Access, are also considered in the project.

Overall Goal: Increased participation of vulnerable people to actively affect and influence development through the integration of Community Engagement and Accountability, Gender and Diversity into SSRC projects, branches and ways of working

Outcome 1: Community members’ participation in program implementation is improved, ensuring their views are used for program decisions

Outcome 2: Strengthen SSRC capacity to measure and report on CEA

Outcome 3: Community members are well-informed about the mandate and role of SSRC

Outcome 4: Increased knowledge of Gender and Diversity, mainstreamed in SSRC and its projects and operations.

3. Evaluation Purpose and Scope

3.1. Purpose (overall objective)

To assess the progress made by the CEA project towards achieving its objectives, intended and unintended results, as well as drawing lessons learned and recommendations for continued implementation across the National Society after the end of the project – in accordance with the intentions of the SSRC Strategic Plan 2018-21 that aims at rolling out CEA across the NS during this time period. The evaluation also aims at building the capacity of the SSRC PMER team to carry out evaluations.

3.2. Scope

Unit to be assessed: The progress of the Community Engagement and Accountability project in South Sudan Red Cross will be assessed. Geographical coverage of the evaluation will include the National Headquarters of SSRC in Juba and selected SSRC branches and communities.

The period of the operation to be reviewed: January 1, 2017 to date (March 2020). Project will end 31 March 2020.The stakeholders to be included in the review include: CEA project team and targeted SSRC branches, SSRC volunteers, PACs (Project Advisory Committees), Community Development Committees, community members and local authorities.

4. Evaluation Criteria – Objectives – Questions

The following evaluation criteria will be assessed in this evaluation, in line with the IFRC Framework for Evaluation[1]:

4.1) Relevance & Appropriateness:

• To what extent are the objectives of the project still valid?
• Is there a need to change program implementation and/or direction when rolling out the CEA approach across the NS without a specific CEA project?
• Are the activities and outputs of the program consistent with the overall goal and the attainment of its objectives?
• Are the activities and outputs of the program consistent with the intended impacts and effects?
• Were the needs assessment and design relevant to the context?
• To what extent did the CEA project contribute to the fulfillment of SSRC Strategies, Policies and guidelines as well as IFRC’s Strategy 2020?

4.2) Efficiency:

• Assess the efficiency of the program in converting its inputs (funds, expertise, time etc.) to outputs, with an indication of whether the program has represented good ‘value for money’ given the resources invested. Could it have been done better, cheaper or quicker?
• Assess the efficiency of program management including South Sudan specific constraints in relation to financial management practices, human resource management systems and tools, development of budgets and work plans (according to annual external audits if possible)
• Assess the extent to which attempts have been made to improve cost-efficiency and reduce administrative costs of program implementation
• Have the objectives been achieved on time/are the objectives likely to be achieved on time?

4.3) Effectiveness:

• To what extent are the objectives achieved to date, in relation to set targets?
• To what extent are the objectives likely to be achieved?
• What were the major factors influencing the achievement or non-achievement of the objectives?
• Apart from achieving the planned objectives, which other programme areas adopted the CEA project, leading to improvements and unintended results?
• What did the CEA project ultimately lead to? Did vulnerable people increase their participation, affect and influence on their own development through CEA?

4.4) Coverage:

• Who was targeted by the CEA project? How many? Provide breakdown relating to gender, age, and vulnerability. Why were these targeted, and why were others not?
• Which demographic/vulnerability criteria have been taken into consideration when selecting the target group of the project (gender, age, minorities/religion, disabled, socio-economic status, etc.)?
  Has the project successfully targeted the selected community members?
• Is there a need to include any other target groups/vulnerability criteria into the future application of CEA?

4.5) Sustainability:

• Follow up and comment on the implementation of the Management Response to the recommendations from the CEA MTR (Midterm Review)
• Which measures have been taken to ensure that the benefits of the project will continue after donor funding has ceased (2020)?
• What are the major factors which could influence the achievement or non-achievement of the sustainability of CEA?
• To which extent will the results of the CEA project contribute to The CEA Africa Roadmap?

5. Evaluation Methodology

The final evaluation will be carried out by an external consultant/company in cooperation with members of the project management team, SRC Country Representative, supported by SRC PMER/CEA advisors. A desk review will review secondary data. Qualitative data will be collected through reflection workshops, capacity assessments of branches, key informant interviews. Quantitative data should be gathered likewise.

All data collection will have to happen both at the SSRC HQ level and in the field.

Documents for desktop review:

• Reports from visits to the project (monitoring/advisor visits etc.)
• Indicator tracking tables
• Project reports, quarterly and annual
• Audit reports and budgets
• Workplans
• Report from CEA ‘Kickstart’ event
• CEA baselines
• BOCA from branches
• Midterm review (MTR)
• Reports from Gender and Diversity workshop

Key informant interviews:

• Staff
• Volunteers
• Community members
• Other key informants

Survey questionnaire:

The Consultant/Company will have to provide a survey questionnaire(s) and a sampling methodology for measuring changes in community resilience and influence on projects in their communities.

The data collected will be both qualitative and quantitative in nature. All data, qualitative and quantitative, collected through the evaluation must be disaggregated by age and sex; that is, separately for girls and boys, men and women. The triangulation of information gathered during the quantitative and qualitative research will be done whenever possible.

The final evaluation will result in a written report in English, describing the methods and limitations, findings, conclusions, lessons learned and a reasonable number of recommendations. To be able to implement the recommendations, they should be elaborated rather than simply being a list of areas that should be improved. They should be directed to the SRC and the SSRC. The report shall contain an executive summary with the major findings of the evaluation, the five key recommendations, and lessons learned. The length of the report should not be more than 30 pages plus annexes.

6. Deliverables (or Outputs) and proposed timeline

The evaluation is expected to last maximum 30 days over the period of two calendar months. (24 February- 25 March 2020)

13 March Presentation of initial findings

20 March Draft report to be shared for comments by SSRC and SRC

25 March Comments to the draft report be shared with the consultant(s) 31 March Final report to be shared with SRC/SSRC

7. Responsibilities of the Evaluation team

• To produce an inception report that demonstrates a clear understanding and realistic plan of work for the evaluation, checking that the evaluation plan agrees with the ToR
• Design questionnaire for the survey and finalize quantitative and qualitative tools, compliant to the GPDR data protection and ICRC Handbook on Data Protection in Humanitarian Action
• Analyze the quantitative data collected from the survey and through the qualitative methods
• Debriefing with the SRC and the SSRC of initial findings through a presentation that includes conclusions, and recommendations before revision and final approval of the final report
• Share draft evaluation report in English as per agreed content with SRC/SSRC
• Develop a final evaluation report and provide a soft copy with 3 hard copies to SRC/SSRC[2]
  8. Evaluation Quality & Ethical Standards

The evaluators should take all reasonable steps to ensure that the evaluation is designed and conducted to respect and protect the rights and welfare of people and the communities of which they are members, and to ensure that the evaluation is technically accurate, reliable, and legitimate, conducted in a transparent and impartial manner, and contributing to organizational learning and accountability. Therefore, the evaluation team should adhere to the evaluation standards and specific, applicable process outlined in the IFRC Framework for Evaluation.

The IFRC Evaluation Standards are:

1. Utility: Evaluations must be useful
2. Feasibility: Evaluations must be realistic, diplomatic, and managed in a sensible, cost-effective manner
3. Ethics & Legality: Evaluations must be conducted in an ethical and legal manner, with regard for the welfare of those involved in and affected by the evaluation
4. Impartiality & Independence; Evaluations should be impartial, providing a comprehensive and unbiased assessment that considers the views of all stakeholders
5. Transparency: Evaluation activities should reflect an attitude of openness and transparency
6. Accuracy: Evaluations should be technically accurate, providing sufficient information about the data collection, analysis, and interpretation methods so that its worth or merit can be determined
7. Participation: Stakeholders should be consulted and meaningfully involved in the evaluation process when feasible and appropriate
8. Collaboration: Collaboration between key operating partners in the evaluation process improves the legitimacy and utility of the evaluation

It is also expected that the evaluation will respect the seven Fundamental Principles of the Red Cross and Red Crescent:

1) Humanity, 2) Impartiality, 3) Neutrality, 4) Independence, 5) Voluntary Service, 6) Unity, and 7) Universality.

Further information can be obtained about these principles at www.ifrc.org/what/values/principles/index.asp

The consultants/consultant company should:

• Have experience with the evaluation of CEA and community involvement in project planning
• Have experience of conducting evaluations in the geographical context
• Have experience in working in insecure contexts like South Sudan or similar
• Have previous experience of conducting evaluations for RCRC Movement
• Be fully fluent in English, orally and written
• Evaluators should have an advanced university degree in Social Sciences or a similar field, relevant for the evaluation

Optionally:

• Have knowledge of local languages, especially Arabic

[1] The IFRC Framework for Evaluation (https://www.ifrc.org/Global/Publications/monitoring/IFRC-Framework-for-E…) includes a total number of 8 criteria to guide what is evaluated in IFRC work. Since the CEA programme is carried out in a complex setting with different policies, policy coherence has to be assessed. Likewise, the impact must be measured in this evaluation.

[2] Three hard copies and a soft copy on USB to be sent to SRC Country Office in Juba via courier and a soft copy via email to SRC Country Representative in Juba, South Sudan)