Evaluation of the Consortium Project “CSOs & POLICY DIALOGUE” in East Africa 385 views0 applications

In order to download a pfd Version of the Terms of References, including figures and a description of involved partner organizations, click here.

1. Introduction/Background

The Consortium Project “CSOs & Policy Dialogue – Further strengthening capacities of CSOs engaging in Policy Dialogue” is a three years programme funded by the Austrian Development Agency (ADA). Its current phase (Jan 1^st 2017 – Dec 31^st 2019) is the second phase of the project succeeding its pilot phase (Dec 1^st 2014 – 30^th Nov 2016) and is implemented by a Consortium of Austrian NGOs and their local partners in East Africa (Kenya, Uganda, Tanzania, Rwanda), with a total project budget of 495.000 EUR.

The design, implementation and results of the project will be evaluated externally in the second quarter of 2019 in order to inform the planning process of its follow-up phase.

About the Project

An independent evaluation of ADA, Danida and SIDA from year 2012 drew important lessons on why Civil Society engagement in policy dialogue in the Global South is not effectively working, especially when it comes to topics relevant for marginalized population groups. The evaluation team recommended to devote more resources to build CSOs’ capacities and to come up with adequate supporting and financing tools for their engagement in policy dialogue .[1] The Austrian Development Agency took on the recommendations of the independent evaluation and invited Austrian NGOs collaborating with CSOs in the Global South to develop a project on CSOs’ engagement in Policy Dialogue.

From Nov 2014 to Dec 2016, a Consortium of 5 Austrian NGOs implemented a pilot project with the objective to strengthen capacities of East African NGOs regarding policy dialogue. In the 2 years of implementing the project, 7 East African organisations were trained on different tools and ways of engagement in policy dialogue, evidence based research and media engagement, while, at the same time, gaining various experiences from implementing small action funds for policy dialogue activities.

The second project phase of the consortium project started in January 2017 and will end in December 2019. It continues with proven approaches and activities by providing tailor-made capacity development of East African Civil Society Organizations (CSOs), financial support of partner organisations for their policy dialogue engagement as well as room for capitalizing experiences and spreading recommendations.

The specific objective of the present project phase is to “Further strengthen capacities of East African CSOs regarding their policy dialogue engagement”. The long term change process the project envisages is that a more effective engagement of Civil Society in Policy Dialogue leads to a stronger reflection of the population needs, especially the ones of the most marginalized, in national and local policies (See Figure 1).

The Expected Results of the Project are the following:

1. Capacity development and cross-learning plan for partner CSOs is developed
2. The ability of East African CSOs to engage in policy dialogue has been further strengthened in their field of work
3. Recommendations/ guidelines for CSOs engagement in Policy Dialogue are further complemented and used/ applied

In order to strengthen the abilities of the partner organizations to engage in Policy Dialogue two main components complement each other:

• The capacity development plan consisting of collective training workshops as well as cross-learning and organization-focused Capacity Development activities is implemented.
• Partner Organizations plan and implement their own Small Action Fund Initiatives, aiming at influencing policy making spaces, and allowing them to develop their own Policy Dialogue Strategies and to test and consolidate different methods and approaches for effective engagement along the Policy Cycle (See Figure 2)

The target group of the consortium project is the staff of 9 selected local partner organizations in East Africa (as well as other East African CSOs who benefit as guest learners from training workshops, experience exchange and shared products in the policy dialogue field). Capacity Development measures are based on a needs assessment at focused CSOs and treat a variety of topics, such as strategy development and risk management for PD engagements, research, documentation, resource mobilization and M&E concerning PD initiatives, stakeholder engagement as well as networking for PD.

The Small Action Fund initiatives for Policy Dialogue of these partner organizations benefit more than 7.000 persons directly, while the envisioned change in policy frameworks benefits potentially up to more than 2 Million persons indirectly. The initiatives are focusing on a variety of topics and regions such as: preventing child marriages and pregnancies in Karangura, Uganda; budget allocations for most vulnerable children in 6 regions in Tanzania; disaster risk management in Marsabit, Kalifi and Lamu Counties, Kenya; mandatory first aid training for licenced drivers in Rwanda, prevention of GBV and land tenure security for women and girls in Northern Uganda; just and fair trade for women farmers in Nakuru and Nyandarua Counties, Kenya; good governance and rule of law in Bungoma County, Kenya.

About the Consortium

The Consortium is composed of the following 5 Austrian NGOs: HORIZONT3000, Red Cross Austria, SOS Children’s Villages Austria, Caritas Austria and CARE Austria. The consortium partners in Austria steer and coordinate project implementation via a steering committee meeting regularly in Vienna to discuss the programme’s progress.

HORIZONT3000 is the lead agency, which coordinates the project and is responsible for implementing, monitoring and reporting towards the main donor, the Austrian Development Agency. The lead agency is also managing all activities on capacity development (needs assessment/ mapping, planning and implementation trainings for local partner organisations and following up on them), experience capitalization as well as sharing events in East Africa.

Further, each consortium partner is responsible for planning, implementing and monitoring small action fund and cross-learning activities of the respective local partner organisations, as well as for reporting to HORIZONT3000 on these activities.

The 9 local partner organizations of the Austrian Consortium who benefit from project components (capacity development), and implement project activities (Small Action Fund Initiatives) are the following: DESECE and GROOTS Kenya (partner of HORIZONT3000), Kenya Red Cross and Rwanda Red Cross (partner of Red Cross Austria), SOS Children’s Village Uganda and SOS Children’s Village Tanzania (partner of SOS Children’s Villages Austria), MIO-NET (partner of CARITAS Austria), CARE Uganda and its local implementing partner WORUDET (partner of CARE Austria). See Annex I for short descriptions on the organizations.

About the lead agency

HORIZONT3000 is an Austrian NGDO with nine catholic member organizations. HORIZONT3000 is mainly funded by the Austrian Government (Austrian Development Cooperation), the European Union, contributions from its member organizations, Liechtensteinischer Entwicklungsdienst (LED), as well as the Medicor Foundation Liechtenstein.

HORIZONT3000’s core business is providing capacity development for local partner organizations. It is specialized in the monitoring and implementation of projects and in sending experts to developing countries for advisory of partner organisations. In 2017, HORIZONT3000 steered 164 projects in 17 countries with an annual budget of 11.15 million Euros. Around 30 people are employed in the HORIZONT3000 office in Vienna and more than 80 experts are working in field offices (in Nicaragua, Guatemala, Uganda, Mozambique, Senegal and Papua New Guinea) and projects of HORIZONT3000 in partner countries.

Its expertise is expressed by high-quality monitoring and project management and a wide experience with co-financing, technical assistance, open and active dialogue with the governments of partner countries and contributing to official development policies.

About the main donor

The Austrian Development Agency (ADA) is the Operational Unit of the Austrian Development Cooperation (ADC). It is in charge of implementing all bilateral programmes and projects in ADC’s partner countries and administers the budget earmarked for this. Another part of ADA’s operations is related to development education and awareness-raising in Austria.

The Austrian Development Agency cooperates with non-governmental organisations (NGOs) and combines official development assistance with numerous civil-society initiatives. This way, government and civil society make a joint contribution to poverty reduction and improving the conditions of life in developing countries.

2. Purpose

The evaluation is intended to contribute to internal learning, the accountability towards our stakeholders and to provide recommendations for a possible future phase of the project.

• Learning: insights and knowledge gained through the evaluation will contribute to the steering and implementation of the ongoing and planning of a follow-up phase of the Consortium Project.
• Accountability: the outcome of the evaluation will deliver insights on how the Consortium Project achieves results. It will be crucially important for the accountability towards stakeholders, especially the Austrian Development Agency.
• Recommendations: The evaluation report will be valuable in terms of recommendations on how to increase positive impact in the field of policy dialogue engagement of local CSOs as well as possible improvements of the management, steering and coordination processes involved in the implementation of the project.

3. Objective

The main objective of the evaluation is to assess the design and implementation, fulfilment of objectives and achievement of expected results of the Consortium Project “CSOs & Policy Dialogue” in East Africa.

4. Focus and Scope

The evaluation shall assess the current phase of the project (01-2017- 12-2019), while also taking into account the pilot phase (12-2014 – 11-2016).

The evaluation shall assess all 3 mentioned results while emphasis shall be put on Result 2: The ability of East African CSOs to engage in policy dialogue has been further strengthened in their field of work by assessing the relevance, effectiveness, efficiency, impact and sustainability of two main aspects of the project:

• a. Capacity development measures benefitting 9 CSOs supported in this project
• b. Small Action Fund Initiatives implemented by 9 CSOs supported in this project

While in a. (capacity development) all 9 partners shall be included in the assessment, in b. (small action funds) a selected number of partners shall be assessed due to budget limitations of this evaluation combined with the vast geographical area the project encompasses: out of 9 organizations, a minimum of 4 (maximum 6) organizations in different locations in Kenya and Uganda shall be visited in order to assess their Small Action Fund initiatives for Policy Dialogue.

5. Main guiding questions

The evaluation shall assess the intervention logic of the project and the relevance, effectiveness, efficiency, impact and sustainability of the project by addressing the following main guiding questions:

Relevance

1. To what extent does the project address the needs and interests of involved partner organizations and their beneficiaries?
2. Are the topics addressed and instruments applied in capacity development measures appropriate to achieve the desired objective of the project?
3. Are the topics addressed and instruments applied in the Small Action Fund initiatives appropriate to achieve their desired objectives?
4. To what extent are the cross-cutting topics gender equality and environment considered in the planning and implementation of activities?

Effectiveness

1. To what extent has the objective of the project been achieved? Which are relevant factors for (not) achieving the objective?
2. Are the capacities created and strengthened used appropriately to improve policy dialogue initiatives of partner organizations and develop their respective strategies?
3. Which capacity development instruments did (not) prove to be effective for improving the performance of targeted CSOs in the policy dialogue field?
4. Which policy dialogue approaches and methods in partner initiatives did (not) prove to be effective to influence policy making process in a meaningful way?
5. How effectively are synergies between partner organizations used? How effective are cross-learning activities for improving the engagement of partner organizations?
6. How effectively were other stakeholders involved and/or addressed in the project (policy makers, media and other CSOs) in order to increase the benefit of the project?

Efficiency

1. Have financial resources and inputs been used efficiently to achieve objectives and results? Are there alternative, more cost-efficient measures to achieve envisioned results?
2. How do participants who have attended training workshops and other capacity development activities transfer the knowledge and skills gained into their organisation?
3. What are the strengths and weaknesses of the project steering and management processes of the consortium project?

Impact

1. To what extent did the Small Action Fund Initiatives of partner organizations affect national and local policy processes in a positive way for marginalized populations?
2. What other impacts – positive or negative, intended or unintended – can be identified because of the project and specifically by the consortium modality of project implementation in East Africa?

Sustainability

1. How far are attained improvements and positive effects at organizational as well as policy level sustainable? Are strengthened capacities anchored in the organizations?
2. What risks and potentials are visible regarding the sustainable effectiveness of the project?
3. To what extent will activities and results be expected to continue after donor intervention ends?
4. To what extent did the project strengthen local ownership and leadership?

6. Methodology

It is up to the consultants, to suggest the methodology and action plan as part of the offer and it is expected that the consultants clearly define the intended approach.

For the desk study, the consultants will be provided with the following documents:

• Application of the consortium project pilot phase (2014) and phase II (2017); final report pilot phase (2016) and yearly progress report phase II (2017);
• Capacity Development Needs Assessment, Capacity Development Plan and reports and survey results of all trainings/workshops organized by HORIZONT3000
• Products of the project meant for dissemination among CSO networks: video on the policy cycle, guidelines, recommendations and a draft manual for CSO engagement in PD.

It is suggested that the following actors, stakeholders and beneficiaries are involved in visits, interviews and/or surveys:

• Project Coordinator of the Consortium Project and all members of the Steering Committee of the Consortium Project (5 representatives of Austrian Consortium partners)
• Project Staff of all 9 Partner Organizations who benefitted from Capacity Development measures and/or are involved in implementing Small Action Fund and other Policy Dialogue Initiatives of the Organization
• Representatives of beneficiaries of Small Action Fund Initiatives of 4-6 selected partner organizations in Kenya and Uganda
• Consultants who accompanied capacity development processes and the elaboration of products for dissemination in a meaningful way.

Data should be collected and interpreted in a gender-disaggregated manner. OECD DAC Evaluation Quality Standards and ADA Evaluation Guidelines are to be applied and the compliance needs to be comprehensible in the evaluation.

7. Deliverables

Besides appointments, meetings and interviews for the data collection according to the methodological approach of the consultants, the plan of activities shall include a (virtual) meeting with the Project Coordinator to discuss the inception report and kick off the data collection process, as well as a (virtual) meeting with the Project Coordinator and, if possible, all members of the Steering Committee after completing the data collection to discuss provisional results.

The inception report shall have a maximum of 10 pages and include a detailed outline of the assessment’s aims, approaches, methods, work plan, surveys etc.).

The consultant(s) shall present a draft report at the latest two weeks after the presentation of provisional results. The draft report will be assessed by HORIZONT3000 as to its comprehensibility and comprehensiveness and the consultants are expected to integrate feedback in the final report.

The final report shall have a maximum of 25 pages (without annexes and index) and include: title page, table of content, list of abbreviations, executive summary (max. 5 pages), background, introduction, methods, findings, conclusions, lessons learnt, recommendations and Annexes (Logframe, terms of reference and schedule of the evaluation, list of key informants, list of documents used, questionnaires or other instruments used; Reports prepared for the field study; Information regarding the evaluators, and other document the consultant(s) consider relevant).

Additionally a visually appealing one-pager shall be developed, representing the main findings of the evaluation at a glance.

8. Coordination

The general coordination of the evaluation including the selection, contracting and feedback processes is assumed by Kristina Kroyer, Project Coordinator Policy Dialogue at HORIZONT3000 Vienna ([email protected])

9. Time Frame

The evaluation shall take place between March and June, with the draft report completed by latest May 15 and the final report completed by latest June 14, 2019.

10. Budget

The total budget available for the evaluation is max EUR 14.000.

This value must cover all costs relating to the assignment including fees, taxes, food, accommodation, national/international transportation, office materials, insurances, etc.;

HORIZONT3000 does not have any additional budget to organize workshops for the evaluation. Workshops must be included in the budget and should be easy to handle by the partner organizations.

Payment of the fee will be made in two tranches:

• · 50% at signing of the contract
• · 50% after submission of final report

Annex: Description of Partner Organizations

Click here, for the full document.

[1] ITAD/ COWI: Joint Evaluation of Support to Civil Society Engagement in Policy Dialogue. Synthesis Report, November 2012.