Terms of Reference End Evaluation of the Every Voice Counts Programme in Somalia 57 views0 applications

Table of Contents

1 Every Voice Counts programme 3

1.1 Background 3

1.2 Focus in country 4

[Please use country specific bullet as intro and add text about the countries ToC] 4

2 The assignment 5

2.1 Objective and scope 5

2.2 Evaluation questions 5

2.3 Methodologies 7

2.4 Timeline and requirements 7

2.5 Roles and responsibilities 7

2.6 Planning 8

3 Purchase procedure 9

1 Every Voice Counts programme

1.1 Background

The Every Voice Counts (EVC) Programme is funded through the ‘Dialogue and Dissent’ strategic partnership programme of the Netherlands Ministry of Foreign Affairs and aims to contribute to inclusive and effective governance processes in fragile settings.

The programme is being implemented by CARE Nederland (CNL), The Hague Academy for Local Governance, RNW Media, CARE Country Offices (COs) and local partners in Afghanistan, Burundi, Pakistan, Rwanda, Somalia and Sudan.

A key obstacle to achieving stability and sustainable development in fragile settings is the structural exclusion of segments of society from local, district level and national governance processes. The EVC programme Theory of Change (ToC) addresses this by focussing on inclusive governance processes, referring to the extent to which governance processes provide space to overcome the systematic exclusion of disadvantaged groups seeking to participate in decision making, in particular women and/or youth (see Annex 1-Theory of Change). https://bit.ly/39og1Ie

CARE believes that inclusiveness leads to more effective governance since institutions and policies will then be accessible, accountable and responsive to disadvantaged groups, protecting their interests and providing diverse populations with equal access to public services such as justice, health and education. The ToC identifies four “domains of change” in which change is required to realize our long-term objective: to contribute to inclusive and effective governance processes in fragile- and conflict-affected settings. The domains are mutually constitutive and change in one can trigger change in the other.

Domain 1 is centred on empowerment of members of excluded groups – in particular, women and/or youth – on lobby and advocacy, as most members of excluded groups are affected by harmful norms and traditions, discriminatory social structures, lack awareness of their rights and responsibilities and have limited capacities to articulate and act on their needs.

Domain 2 focusses on strengthening the lobby and advocacy role of civil society as it has a key role to play in influencing policies and practices on behalf of excluded groups and in holding public authorities and other power holders to account.

Domain 3 is about responsive public authorities and other power holders. We believe that responsiveness increases when public authorities and other power holders have stronger capabilities as well as stronger incentives to act upon the needs and interests of excluded groups.

Domain 4 focuses on expanding and strengthening formal and informal space for dialogue and negotiation. We believe that increased engagement and effective interaction between all actors leads to policies and practices that better take into account the interests of excluded groups.

Figure 1. Provides a graphic representation of the EVC programme ToC.

4. Effective Spaces for dialogue and negotiation

Long-term Objective

Impact

Domains of Change

Stability and Development

Figure 1. Theory of Change Every Voice Counts Programme

1.2 Focus in country (Somalia)

Introduction

· Somalia is affected by two decades of conflict that have exacerbated and gendered extreme poverty and vulnerability, in particular affecting women and with a clear impact on gender roles and norms of both women and men.

· Somalia is divided into five distinct regions, Somaliland, Puntland state, South Central, Jubaland and South West Administration.

· EVC Somalia focuses on inclusive governance in Puntland and South Central Somalia states, in particular young women and men participation in political processes such as the district elections, state elections, federal elections and federal constitution.

· The interventions aim to address exclusion of young women and men (ages 14-29) from political processes and decisions that affects their lives

Theory of Change

· EVC Somalia recognizes that the interests and needs of young women and men may at times differ. The interventions were designed with this is mind and in such a way that ensures young women will be able to participate as effectively as young men. This mean that certain interventions were organized in mixed groups and others in same-sex groups.

· The theory of change is premised on the assumption that; IF members of excluded groups are empowered and are actively influencing decisions that affect their lives,

AND IF Civil society organizations are effectively influencing policies and practices on behalf of excluded groups and are holding public authorities and other power holders to account, AND IF public authorities and other powerholders are responsive to and act upon the needs and interests of excluded groups, and if excluded groups, civil society organizations, public authorities and other power holders effectively interact through formal and informal spaces for dialogue and negotiation, THEN governance processes in Puntland and South West state will be inclusive and effective hence resulting processes and policies are likely to yield expected development results, which increases public authorities’ legitimacy.

· EVC Somalia aims to ensure young women and men in Puntland and South Central Somalia are represented, are able to have their voices heard, and influence decision-making that affects their lives through participation in informal and formal governance mechanisms and decision-making processes, including the federal election and the finalization of the federal constitution.

Impact and process indicators

Long-term objective: Governance processes in fragile settings are inclusive and effective

· % increase of members of excluded groups reporting an improvement in the realisation of their rights

· Reduced number of conflicts related to the exclusion of specific groups

Domain 1: Members of excluded groups are empowered and are actively influencing decisions that affect their lives

· % increase of members of excluded groups stating they have more influence in decision making processes that concern them.

· % increase of members of excluded groups participating in local development planning and the management of basic services and resources.

· % reduction of discrimination (on the basis of gender, age, religion or ethnicity) reported

· % of members of excluded groups who report an increase in understanding of their civil rights and responsibilities

· Increased # of initiatives undertaken by members of excluded groups to advocate for their rights

Domain 2: Civil society organizations are effectively influencing policies and practices on behalf of excluded groups and are holding public authorities and other power holders to account

· % of members of excluded groups expressing satisfaction with the way their civil rights are promoted by CSO’s

· Increase in # of advocacy initiatives undertaken by CSO’s to address the rights and interests of excluded groups

· % of CSO’s which show an increased score on all of the five capabilities of the “5c model”.

· Increased # of members of excluded groups report to participate in and be represented by CSO’s

· Reduced # of reports of restrictive measures in the civil society operating environment

· Existence of CSO legislation respecting the fundamental freedoms of expression, association and peaceful assembly, and allowing for CSO engagement in governance processes, and lobby and advocacy.

Domain 3: Public authorities and other power-holders are responsive to and act upon the needs and interests of excluded groups

· % increase of members of excluded groups stating that local and national policies are addressing their identified needs

· % of targeted members of excluded groups indicating an increased level of confidence in public authorities and other power holders

· Increased # of elected representatives are from excluded groups

· Accountability mechanisms (such as high courts, ombudspersons, auditor general, anti-corruption bodies, elections, budget monitoring, public hearings) exist and are increasingly evaluated positively.

· Increase in # of policies, laws, budgets and action plans that were developed through inclusive mechanisms.

· % of public authorities and power holders maintaining collective assets (water/education/health) that receive a higher score on their scorecard by their users

Domain 4: Excluded groups, civil society organizations, public authorities and other power holders effectively interact through formal and informal spaces for dialogue and negotiation

· Increase in # of agreements and commitments reflecting the interests of excluded groups, as a result of the spaces for negotiation

· Increase in # of functional spaces and mechanisms for institutionalised participation in policy formulation and planning processes

· Members of excluded groups, CSO’s, public authorities and other power holders indicate awareness of relevant spaces for negotiation and that they are increasingly accessible to them.

· All actors express increased satisfaction with the interaction through spaces for negotiation

These impact and process indicators are not conclusive – the evaluation will also be guided by the programme adaptations and adjustments to the theory of change taking into cognisance the various change pathways and issues “tracks” adopted based on context.

Worth noting is that a baseline study was carried out in June 2016 to establish the situation at the start of the EVC programme enabling the assessment of change over time. A midterm review was conducted in September 2018 to assess progress against baseline values.

2 The assignment

2.1 Objective and scope

The aim of this assignment is to conduct the End Evaluation of the Every Voice Counts programme in Somalia as input for the Global Evaluation Report. The EVC programme will be evaluated in its entirety, covering the period 1 January 2016 – 1 May 2020.

The EEE should comprise:

1. An outcome evaluation – resulting from a synthesis of the 6 country specific reports – focusing on the following programme aspects:

· Effectiveness

· Efficiency

· Relevance

· Sustainability

· Impact

1. Key Lessons Learnt and Actionable recommendations on the design and implementation of an inclusive governance programme in Somalia.

2.2 Evaluation questions

The end evaluation should provide answers to the following evaluation questions:

1. Effectiveness

· To what extent were the EVC programme objectives as defined in the ToC achieved?

Key sub-questions domain 1:

o To what extent are women and youth more aware of their rights?

o To what extent are women and youth engaging in advocacy?

o To what extent have perceptions on accountability transparency and inclusive decision-making changed?

o To what extent have norms related to social inclusion changed?

Key sub-questions domain 2:

o To what extent were capacities of first and second tier CSOs strengthened?

o To what extent are first and second tier CSOs carrying out advocacy initiatives?

o To what extent have CSOs improved their legitimacy to lobby and advocate for the claims of women and/or youth?

Key sub-questions domain 3:

o To what extent were capacities of public authorities and powerholders strengthened.

o To what extent are public authorities and powerholders responsive to the needs and rights of women and/or youth?

o To what extent has accountability and transparency of public authorities and powerholders increased.

Key sub-questions domain 4:

o What type of spaces for dialogue were created and how did dialogue processes take place?

o To what extent were laws and policies improved and implemented?

Key sub-question overarching goal:

o To what extent are decision-making processes more inclusive of women and/or youth?

· What were the major factors influencing the achievement or non-achievement of the objectives?

· Are the two key assumptions underlying the ToC that were formulated in 5-year country plan valid?

o Assumption 1: When interests of excluded groups are represented, resulting processes and policies are likely to yield expected development results, which increases public authorities’ legitimacy – a key factor in securing stability.

o Assumption 2: Accountability interventions shift power relations in favour of excluded groups. Responsiveness increases when public authorities and other power-holders have stronger capabilities as well as stronger incentives to act in these ways.

2. Efficiency

· Which key advocacy interventions are most effective based on the perceptions of relevant stakeholders?[1]

3. Relevance

· To what extent are the objectives of the country programme ToCs still relevant for the targeted women and youth?

· Which adjustments should be made in a next phase of the programme?

4. Sustainability

· What is being done to ensure sustainability of programme results?

· Are these efforts likely to influence the sustainability of the programme?

5. Impact

· What has happened as a result of the programme?

· To what extent has the programme resulted in unintended positive and negative changes?

6. Key lessons learnt and actionable recommendations for future programming

· What are the key lessons learnt during the design and implementation of the inclusive governance programme

· What are the main conclusions and recommendations of this evaluation when it comes the design and implementation of future inclusive governance programmes?

The questions above are not exhaustive, and will still require adjustments to include specific and strategic questions that address all the impact and process indicators listed under each domain

2.3 Methodologies

To assess the effectiveness of the programme, standardised indicators and tools need to be used. An overview of the standardised indicators is provided in Annex 2 ( https://bit.ly/38mG2aQ ) along with the tools to be used. Note that a Global Evaluation Team is responsible for aggregating results of the six participating countries into a global EVC Evaluation Report. The Local Evaluation Team will have to adhere to the timeline and requirements described in paragraph 2.4 to allow of aggregation of results by the Global Evaluation Team. Important inputs from the CARE Country office into the reports at country level are: 1) monitoring data, 2) outcomes harvested using the Outcome Harvesting methodology and 3) Results from vignette studies on social norms.

2.4 Timeline and requirements

As the final evaluation report will be used as input for the Global EVC Evaluation Report synthesizing six country specific evaluation reports, the assignment entails the following:

· Participation in a 4-day workshop on the evaluation framework and required evaluation methodologies: March 2020

· Working with standardised guidelines and tools for data-collection

· Working with a standardised analytical framework

· Training of enumerators: beginning of May 2020

· Data-collection and analysis: mid-May to mid-August 2020

· Organise workshop to jointly[2] analyse and triangulate findings: mid-August 2020

· Organise workshop to validate results: end of August[3] 2020

· Draft evaluation report ready: 1 September 2020

· Final evaluation reports ready: 1 October 2020

The timelines are tentative but the consultant should submit a work plan for thirty (30) days – all inclusive

2.5 Roles and responsibilities

The assignment in this ToR will feed into the global EEE of the multi-country Every Voice Counts Programme. The global EEE of the Every Voice Counts Programme will be conducted by a Global Evaluation team. This team will synthesize the results of 6 country-specific evaluation reports that will be conducted by six Local Evaluation Teams. These Local Evaluation Teams will be hired, managed and reimbursed by CARE Country Office staff and guided and supported by the Global Evaluation Team. CARE Country Office staff is required to provide Local Evaluation Teams with data specified in paragraph 2.3. The EEE is commissioned and managed by an Evaluation Management Team headed by CARE Nederland’s PMEL coordinator. The evaluation will be overseen by a Reference Group comprised of the EVC programme coordinator, CARE Nederland’s PMEL coordinator and experts working in the development sector, academia and the Netherlands Ministry of Foreign Affairs. The mission and purpose of the Reference Group to ensure the final evaluation of the EVC programme adheres to the IOB minimal standards for the end evaluation by providing advice and guidance at pivotal moments in the evaluation process.

2.6 Planning

Deliverables

Time period

Participate in a 4-day workshop on the evaluation framework and required evaluation methodologies (team leader only)

March 2020

Training of enumerators

May 2020

Data-collection and analysis

mid-May 2020 to

mid-August 2020

Organise workshop to analyse and triangulate findings

mid-August 2020

Organise workshop (including the Dutch embassy) to validate results

End of August 2020

Draft evaluation report ready and raw data available for Global Evaluation Team

1 September 2020

Final evaluation report ready[4]

1 October 2020

3 Purchase procedure

Please use standard text from Country Office. Note that we aim to have local teams hired by mid- February 2020 to allow for time to arrange visa for the workshop in March 2020 which will take place in the Netherlands.

[1] The basic efficiency resource methodology should be used see: Cugelman B. and Otero E. (2010) Basic Efficiency Resource: A framework for measuring the relative performance of multi-unit programs. Leitmtoiv and AlterSpark.

[2] With staff of CARE and partners

[3] This validation workshop should include staff from the Dutch Embassy.

[4] Note that both the draft and final evaluation report need to adhere to the standardized format provided by the Global Evaluation Team.