Mercy Corps seeks to undertake research to better understand the effect of youth educational programming on stability in Somalia. Through a mixed-methods evaluation of the Somalia Youth Leadership Initiative (SYLI), a $38 million USAID-funded youth development program, this study would assess the degree to which improved access to education and opportunities for civic engagement can contribute to stability in Somalia.
Access to secondary education in Somalia is among the worst in the world with only 52,000 students enrolled in 195 secondary schools (64 public schools). Enrollment rates are highest in Somaliland (16,410 students in 42 schools), less in Puntland (6,773 in 22 schools), and lowest in Galmudug (1,407 students in 10 schools) and other parts of south central Somalia. Only 2-6% of the 2.7 million Somali youth (ages 14-17) are enrolled in secondary school. 10-20% of the student population is female with numbers even lower in rural and pastoralist communities.
SYLI targets youth age 15-24 in the Somaliland, Puntland and South Central Somalia and will directly benefit more than 150,000 youth with the potential to reach and impact millions of Somali youth, educators, and communities through radio, media and advocacy campaigns.
The proposed study seeks to answer the following research questions:
- What, if any, is the effect of increasing access to secondary formal education on in-school youth’s attitude and behaviors on violence in Somalia?
- What, if any, is the effect of civic engagement activities for out-of-school youth’s attitude and behaviors on violence in Somalia?
- What, if any, is the effect of both education and civic engagement on in-school youth’s attitude and behaviors on violence in Somalia?
- Which of the two interventions (secondary formal education or civic engagement) or a combination of both has a greater impact on reducing violence?
To assess the impact of the SYLI Program on the abovementioned outcomes, we will conduct surveys in areas of Central Somalia where SYLI has increased access to formal secondary education through the construction or reconstruction of schools and undertaken civic engagement activities as well as neighboring communities, which have yet to implement SYLI activities. Within program-targeted communities, we will sample youth (15-24yrs) in two categories: youth currently in Mercy Corps constructed schools and out of school youth who are participating in Mercy Corps civic engagement activities. Amongst those surveyed within SYLI schools, we will differentiate between youth who have participated in civic engagement activities with those who have not to see if civic engagement has an added effect on the intended outcomes when coupled with access to education. In non-program targeted communities, we will sample youth neither in schools nor engaged in civic engagement activities, which will serve as a pure control group. In doing so, this design will allow us to assess the impact of education vs. civic engagement vs. education + civic engagement vs. neither education nor civic engagement. Areas for data collection will be determined after an assessment of the security situation in areas where Mercy Corps is working in South Central Somalia.
Subject to resource constraints, an overall sample size of 1,800 youth will be targeted in the survey. The selection of survey respondents will follow the following approximate approach:
- A set number of Primary Sampling Units (PSUs) or clusters will be selected randomly, out of a total set of PSUs defined as communities in which Mercy Corps has implemented the SYLI program in South Central Somalia.
- For the comparison group (non program-targeted) communities, we will pair each selected PSU in which the SYLI program is taking place, with a neighboring community that has similar socio-economic and demographic features, using a matching technique. We will rely on census data collected by agencies such as the UNFPA for the purpose of matching to minimize selection bias.
- Within each community, we will randomly select youth to participate in the survey. If a household list is unavailable, we will rely on other methods, such as community mapping to develop a sampling frame of youth. We will use quota sampling to ensure that we have adequate representation of the categories of youth described above.
Purpose / Project Description:
The objective of this impact study is to:
- Generate credible evidence on the attributable impact of access to education and civic engagement activities through the SYLI program on stability outcomes within Somalia.
- Contribute to understanding of the opportunities and limitations of educational and youth development programming like SYLI in promoting stability in a context like Somalia.
Consultant Objectives:
To contribute to the impact study, we require the surveys of an experienced firm to lead the administration of a standardized survey. The objective of the survey is to lead the collection of high quality data from a sample of youth in Somalia, and deliver a clean, complete dataset that is ready for analysis.
Consultant Activities:
The consultant will carry out the following activities based on the design, data collection methodology and implementation steps as provided by Mercy Corps:
- Participate in a research and survey design workshop for an estimated 2 days at least 2 weeks before the beginning of data collection (exact date to be confirmed) to discuss and finalize sampling plan and methodology, including selection of the specific locations in which survey work will be done.
- Translate the surveys. The Survey Firm will be provided the questionnaire in English. They will translate this set of questions into Somali, adapting any components that are necessary to accurately capture the intended information on the study populations.
- Code questionnaire and upload forms into ONA or other mobile data collection platform. Coding strategy should ensure each individual surveyed has a unique code for data entry that merges seamlessly across data sets. If data collection is done using paper questionnaires, develop data entry template with built in quality control measures
- Internalize and field test of the survey instrument Report back to Mercy Corps any changes required to the survey, and incorporate those changes, once agreed upon with the research team.
- Develop survey fieldwork plan, including transport and other logistics.
- Recruit survey team, design and implement training for the survey team:
- Recruitment of sufficient number of qualified data collectors with appropriate language skills. It is recommended that the data collection field staff have prior experience in similar survey undertakings.
- Preparation of training plan, field work protocols according to agreed methodologies
- Implementation of training for data collectors and supervisor/verifiers, including responsibility for logistics, venue, equipment, etc.
- Training for data collection team on survey questionnaire including step-by-step instructions on data collection and guidance on handling problems and questions that might arise during data collection; additionally, training on mobile data collection should be incorporated for any data collectors without prior experience.
- Ensure that all survey team members have up-to-date knowledge about selected data collection methodology, interviewing techniques, household and respondent selection process, and the completion of the survey and refusal forms.
- Ensure all survey team members have a letter of authority to show both respondents and local authorities as necessary to facilitate their work.
- Implement data collection among the full sample, supervise field data collection and verification:
- Deployment of data collectors and supervisor/verifiers to the field
- Prepare and ensure the delivery of survey materials and equipment
- Oversee interviewers’ works while in the field for timely survey data quality assurance
- Check completed questionnaires, comparison, validation and ensure the proper administration
- Verify a sample number of the completed questionnaires after data collection. Conduct verification of data at least 5% of the total number of respondents.
- Data collection of 1,800 questionnaires (50% from from Mercy Corps intervention zones in Somalia, 50% from non-targeted control group communities) taking into account guidelines on: a) target population and sampling; b) data collection methods and tools used; c) data collection team; d) data verification.
- Data synthesize, cleaning and coding and computer entry
- Clean data for irregularities and misentries
- Code all variables in survey in user friendly format
- The Consultant will deliver the final, cleaned dataset of the survey to Mercy Corps along with a codebook
- After completing all data collection and computer data entry, the Consultant will produce a brief report (in English) to Mercy Corps. The report will describe the preparation process, methodology data collection experiences, verification and entry process, including limitations, challenges.
- Mercy Corps will be responsible for data analysis and production of the impact evaluation report.
Consultant Deliverables:
The Consultant will:
- Input on the Sampling Design
- Provide a finalized survey fieldwork plan
- Translate survey questionnaire to Somali
- Provide data from survey pre-test/pilot
- Provide training to enumerators on survey tools and questionnaires
- Provide weekly updates of activities during data collection
- Provide a final cleaned database of survey data in English
Timeframe / Schedule:
The Contractor will conduct the endline survey in February-March 2017 with a focus on data collection from 1,800 individuals in South Central Somalia). In addition, the contractor will perform data coding and computer data entry. Presumably, a maximum of nine calendar weeks will be required for the assignment, which will be finalized through discussion between both parties. It is expected that the whole task should be finished within sixty three days from the starting date as mentioned below:
Week 1&2
- Participate in design workshop
- Review survey instruments
- Update translations of questionnaire into Somali
- Consult with SYLI key staff, produce and submit fieldwork plan
Week 3
- Produce and submit survey plan
- Selection and hiring adequate number of members for the survey
Week 4
- Training of survey team members, questionnaire field test etc.
Week 5, 6 &7
- Implement data collection and supervise field data collection
Week 8
- Data verification and cleaning questionnaires
Week 9
- Synthesize, coding and cleaning of data
- Produce implementation report
The Consultant will report to:
SYLI Chief of Party
The Consultant will work closely with:
Mercy Corps’ Research and Learning Team, based in Washington D.C.
Application process
Interested candidates who meet the required qualifications and experience are invited to submit their Expression of interest (EoI). The EoI should include:
- CV (s) with details of qualifications, experience, telephone number and names of three referees
- Reports for similar consultancies done
- Technical proposal which must cover: overall concept and approach, methodology and data collection techniques including ensuring data quality, data coding and data entry process, and field operational plan
- The foreseen work plan and time frame for the days worked
- Financial proposal with justification and explanation of any costs included
Evaluation and selection criteria
The successful candidate will demonstrate:
- At least five years of experience in survey and related research work.
- Prior experience with international donor-funded project in conducting large survey (>1,000 respondents) or research related activity in Somalia or the East Africa region.
- Appropriate human resources, financial management capacities, and technical expertise to implement proposed activities.
- Appropriate understanding of relevant survey process, tools, methods, data verification and analysis related issues.
- Previous experience carrying out economic and social project surveys.
- Experience with attitudes and behavior surveys.
- Experience with and authorization for using mobile/hand-held technologies and GPS devices (strongly preferred).
- Somali and English language communication skills of the survey team.
