Tipping Point Programme Evaluation 66 views0 applications

Request proposals for a full-service research agency to design, run and deliver an evaluation for Girl Effect’s Tipping Point programme in Kenya.

Who We Are

Girl Effect is an international non-profit that builds media that girls want, trust, and need. From chatbots to chat shows and TV dramas to tech, our content helps adolescent girls in Africa and Asia make choices and changes in their lives.

We create safe spaces for girls, sharing facts and answering questions about health, nutrition, education, and relationships, empowering girls with the skills to negotiate and redefine what they are told is possible “for a girl.”

Our reach is 50 million and counting and we’re using technology to reach girls at scale so every girl can choose to be in control of her body, her health, her learning, and her livelihood.

Because when a girl unlocks her power to make different choices that change her life, it inspires others to do so too. She starts a ripple effect that impacts her family, community, and country.

That’s the Girl Effect.

Background

Girl Effect has recently launched a program that aims to shift gender norms and drive behavior change for adolescent girls and young women (AGYW) around Sexual and Reproductive Health (SRH) and Women’s Economic Empowerment (WEE).

Working closely with young people in Nairobi and Migori counties, we will design a multi-media behavior change ecosystem encompassing diverse content formats such as social media, influencer campaigns, radio broadcasts, podcasts, and captivating drama series aimed at shifting gender norms.

This multi-media ecosystem will be complemented by a suite of digital engagement tools, including chatbots, IVR, USSD, etc. It will be accessible through high-fi and low-fi digital platforms and in-person interactions facilitated by our network of girl champions to cater to users with basic feature phones or limited access to technology. This intervention will be implemented in Migori and Nairobi, Kenya.

Elements of the intervention will also target influential members of the social networks surrounding girls and young women, including parents, male peers, and partners.

The Scope

Girl Effect is looking to commission a research agency in Kenya with experience in girl-centered qualitative and quantitative research to run an evaluation of Tipping Point. The evaluation should be delivered against the evaluation objectives below. The successful agency will be able to provide local expertise and apply its experience to evaluate the impact of Tipping Point against outcome indicators.

Evaluation objectives

These are our initial evaluation objectives, which will be confirmed with the selected agency.

1. To evaluate the effectiveness of the Tipping Point programme in shifting the behaviors and intentions of young women related to sexual reproductive health (SRH) and women’s economic empowerment opportunities (WEE).
2. To evaluate the effectiveness of the Tipping Point programme in shifting the psychological drivers that contribute towards building intention to act and contribute towards behavioural change (as outlined by Girl Effect’s theory of change).

• Knowledge
• Attitude
• Outcome Expectation
• Perceived control
• Perceived support

1. To evaluate the effectiveness of the Tipping Point programme in shifting the psychological drivers of the social network surrounding adolescent girls and young women.
2. To assess the additional value of on-the-ground interventions alongside our media products and content.

Methodology

Girl Effect strongly supports rigor, reliability, and replicability in its research. We welcome recommendations on the best possible approach to deliver against our research objectives and would like to receive your approach design as a core component of the proposal. Our suggested approach is below. We encourage agencies to suggest adaptations or additional components that would allow for a robust evaluation of the intervention. The final methodology for this evaluation will be designed with the successful agency after they are commissioned.

We envisage a mixed method study. The quantitative and qualitative research will need to capture three cohorts of consumers:

• “Digital consumers” who primarily engage with the content online via social media channels.
• “Low-fi consumers” who attend screenings of the content in their local area.
• “Champion consumers” view content via our ‘Champions’ who visit their local community.

1. Quantitative research

Our priority for this evaluation is to collect quantitative data to assess the effectiveness of our intervention.

We envisage a 6-month long research approach that includes the following:

• A baseline survey, before the intervention, is delivered to assess behaviors, knowledge, and attitudes related to programme outcomes.
• A mid-line survey will be conducted to identify how many consumers have engaged with Girl Effect’s content and any early shifts in behaviours, intent, or drivers.
• An end-line survey will be conducted after consumers have engaged with the content. Participants will repeat the baseline survey to measure the impact of the content.

We encourage agencies to suggest a methodology that will provide a robust evaluation of the impact of our content on all three cohorts. As part of the proposal, we would like agencies to describe how they will design the sample. Please give details on sample size and recruitment selection processes.

1. Qualitative research

We require a qualitative component to the evaluation designed to aid the interpretation of the quantitative data and to gather compelling stories of impact. We envisage this taking place at two points – at the evaluation midline and end-line. It should be a minor and cost-effective component alongside the major quantitative research. Please provide the costs for this component separately.

We envisage this component including a series of focus group discussions (FDGs) and/or in-depth interviews (IDIs). We envisage the agency needing in-person fieldwork with “low-fi consumers” and “champion consumers.” The agency may be able to do online fieldwork with some “digital consumers.”

Girl Effect will be able to connect the agency with some “champion consumers.” However, the agency will need to identify “low-fi” and “digital consumers.”

1. Support network

As part of the study, we will want to speak with priority groups in girls’ social networks. As the programme is further defined, we will be able to provide more details of this support network audience. Still, it is likely to be with either partners or parents and, on a small scale, with a broader range of people in their social network, such as community leaders, teachers, experts in the field, etc. Ideally, this will contain quantitative and qualitative components, but we are open to recommended methodologies to include these groups in a cost effect and robust manner.

Sampling Frame

The respondents of the evaluation must be members of Tipping Point’s target audience: females aged 15-24 years from Migori and Nairobi. The respondents should be a mix of income level, education status, employment status, etc.

After commissioning, our final sample breakdowns will be agreed upon with the selected agency. We would like the sample to allow for statistical analysis of the following age breakdowns: 15-19 years and 20-24 years. And explore any differences that fall out naturally, such as income level, education status, employment status, location (Migori, Nairobi), etc.

We would also like the sample to include influential members of the social networks surrounding girls and young women. We will confirm these groups once we have selected the agency. They may consist of male peers, parents, and partners.

Tasks

The successful agency will be responsible for the following:

1. Design:

• Design the evaluation to meet Girl Effect’s requirements. We anticipate this will likely include the abovementioned component but are open to other recommendations that deliver on objectives.
• Agree on the final design with Girl Effect’s evidence and insight team
• Design sample specification to be representative of target audiences
• Achieve required ethical approvals and obtain approval from relevant public entities
• Design research tools based on Girl Effect’s ToC and previous impact evaluations
• Translate (and back translate) and contextualize the data collection tools into local languages

1. Setup:

• Recruit a suitable team of female researchers and field enumerators for gender-matched interviews with young women and girls.
• Facilitate training for all researchers, including a pre-testing exercise under actual field conditions.
• Complete survey testing and scripting (pilot)
• A post-pilot debrief and working sessions to refine and adapt the data collection following the pre-testing
• Sample and recruit communities and participants according to the provided purposive sampling criteria
• Obtain informed consent from all participants, consent from parents/caregivers, and assent from minors.

1. Fieldwork:

• Capture and store all survey data using digital devices
• Compile and submit field summaries with a breakdown per locality on agreed-upon frequency
• Ensure all work is conducted according to the Girl Effect Girl Safeguarding Policy and an agreed safeguarding agreement
• Conduct data collection in the selected districts. All field worker management, logistics, etc, shall be the agency’s responsibility
• Follow strict data quality assurance procedures and ensure high standards of data management

1. Analysis and reporting:

• Provision of raw and clean data sets
• Provide data tables with cross breaks pre-agreed with Girl Effect.
• Conduct analysis
• Deliver draft reports for Girl Effect’s feedback and input (baseline, midline, end-line and qualitative)
• Deliver final reports for Girl Effect with video-call debrief (baseline, midline, end-line, and qualitative)
• The final report will be delivered to ethical approval boards as required.

The agency will coordinate closely with Girl Effect to deliver work that delivers on all of Girl Effect’s reporting requirements. The Girl Effect Teams will conduct quality assurance during the fieldwork implementation. And review and agree on all major stages of the design and delivery.

Deliverables:

• Evaluation Design (with GE’s support and input)
• Participant sampling
• Survey tool
• Qualitative interview/focus group tools
• Consent and Assent forms
• Translations of tools into relevant languages and formats
• Risk assessment
• Ethical approval certificates
• Training plan
• Raw data files for each stage (baseline, midline, end-line, qual)
• Clean data files for each stage (baseline, midline, end-line, qual)
• Data tables as per GE specification, including sig testing for each stage (baseline, midline, and end-line)
• Baseline report
• Midline report
• Endline report
• Qual component report
• Ethical approvals final report
• Data quality report

Location

The agency must be based or have substantive operations in Kenya and must be conducive to working with teams in the UK, Sub-Saharan Africa & India. The Assignment is to be carried out in Kenya (Migori & Nairobi).

Management

The agency will have a primary point of contact with our Kenya Evidence and Insights Team. The language of the materials and reports shall be English. The contract supervisor shall approve all deliverables submitted by the agency before any payment is made.

Project Timelines

• Agency procurement completed – April 2024
• Ethical approval submitted – June 2024
• Ethical approval achieved – July 2024
• Baseline fieldwork – August 2024
• Baseline reporting – September-October 2024
• Content running from September 2024 to February 2025
• Endline fieldwork – March 2025
• All final reporting – April 2025

Who You Are

Skills Experience and Expertise:

• Extensive experience in designing and conducting quantitative and qualitative research
• Capacity to deliver all areas expected of this research, from design through recruiting and training researchers to leading the data collection process in target field locations, ensuring quality assurance that aligns with research ethics principles, and delivering high-quality reporting.
• Experience designing and conducting quantitative studies in Kenya, especially with adolescent girls, preferably in multiple regions.
• Experience conducting sensitive research with hard-to-reach, most vulnerable populations
• Experience in interviewing minors and adolescents and safeguarding policies and procedures
• Excellent and demonstrated understanding of Child Protection/ Safeguarding and ethical issues in research
• Excellent fieldwork supervision and data quality control strategies
• Ability to conduct a risk assessment and mitigation plans for research
• A fieldwork team of interviewers
• A fieldwork team of female interviewers for girls and young women interviews
• Proven record of effectively managing relationships with research partners
• Excellent interpersonal skills and demonstration of effectively managing relationships with research partners
• Excellent interpersonal skills and demonstration of effectively managing relationships with research partners
• Exceptional communication and organization skills
• Ability to respond to comments and questions in a timely, appropriate manner
• Experience in delivering outputs on time and budget
• Ability to write clearly and concisely in English
• The reputation of the responding agencies and any previous experience in similar surveys/research will also be considered.

Proposal Submission

Interested agencies are asked to submit the following to support their candidature (Max 10 pages):

Technical Proposal:

• Your understanding of the brief and why you feel you are well-placed to advise us on this
• Credentials – Please showcase any previous work that you consider relevant to what we are trying to achieve, how we approach our work, and the audience we cater to. Describe the qualifications, experience, and capabilities of the firm or consultant in providing the requested services (CV/Profile).
• An initial proposal for the design
• Workplan with clear timelines
• Reference – Provide at least three references for similar contracts with a description of the service provided, the value of the contract, and the contract periods of performance

In their technical proposal, the bidder must demonstrate an understanding of the requirements described in the RFP and demonstrate how the bidder will meet the requirements of the evaluation criteria. The technical proposal must be at most ten pages.

Financial Proposal:

• A breakdown of the financial proposal in USD indicating the daily rate for each of the proposed experts, time input, and all applicable reimbursable expenses
• All applicable taxes should be quoted separately.
• Separate costs for key elements of the proposal, i.e., quantitative and qualitative

Technical and Financial proposals will need to be submitted as separate documents. Financial bids will not be opened until the technical evaluation and only for those proposals deemed qualified and responsive. The Proposals shall be prepared in English Language.

GE is not liable for any cost incurred during the award/contract preparation, submission, or negotiation of the award/contract. All submitted documentation and/or materials shall become and remain the property of GE.

The proposal’s VALIDITY shall be 90 days from the bid closure date.

Evaluation Criteria

The criteria against which proposals will be evaluated are listed below.

Technical Evaluation:

• A well-written capability statement clearly outlines your experience in delivering the Scope of Work and how you meet the ‘Who You Are’ requirements above – 20%
• An initial proposal for design (to be finessed with the GE team upon contracting) – 20%
• Ability to achieve project goals/deliverables, i.e., does the proposal have strong feasibility in moving forward with the critical deliverables on schedule? -10%
• Demonstrate geographic experience in Migori and Nairobi (Kenya )- 10%
• Clear and Concise Profile/CV demonstrating relevant expertise – 10%
• Evidence of a minimum of three contactable references – 10%

Financial Evaluation:

• Value for money/proposed budget breakdown – 20%

Procurement Timeframe

• Terms of reference published: 21st March
• Deadline for Questions/Clarifications: 27th March
• Deadline for responses from GE: 1st April
• Proposal submission: 8th April 2024
• Supplier selection, contracting, and briefing: Mid April 2024
• Project commencement: End of April 2024

Ethics

• The successful agency must adhere to Girl Effect protocols and safeguarding measures during all stages of research. This will ensure all girls’ participation will be conducted safely and securely.
• Ethical approval must be submitted to the relevant ethical board to ensure the research takes place with the proper permissions.
• Consent must be obtained from girls with consent forms that clearly state that at any time, research participants are free to decide to leave the research should they feel a reason to do so.
• Care also should be taken by the successful agency to maintain the confidentiality of the information provided by the respondents during the community. Informed voluntary consent should be taken before starting from the community, and parent/caregiver consent should be obtained for a minor. If an individual refuses to participate, they should not be compelled to participate or demoralized by any means. These cases will be treated as no-response under this survey. The research team must all undergo safeguarding training per Girl Effect’s safeguarding procedures and sign the Girl Effect safeguarding policy.
• All data must be stored in password-protected electronic files and shared with Girl Effect via a secure file transfer protocol (FTP)

Tax

Girl Effect is obliged by the Kenyan tax authorities to withhold taxes on service contract fees as well as ensure that 16% VAT, is charged where applicable. Applicants are advised to ensure that they have a clear understanding of their tax position regarding provisions of Kenya tax legislation when developing their proposals.

Copyright

All materials/documents arising from this consultancy work shall remain the property of Girl Effect.

Disclaimer

GE reserves the right to determine the structure of the process, number of short-listed participants, the right to withdraw from the proposal process, the right to change this timetable at any time without notice and reserves the right to withdraw this tender at any time, without prior notice and without liability to compensate and/or reimburse any party. GE shall inform ONLY successful applicant(s). The process of negotiation and signing of the contract with the successful applicant(s) will follow.

Safeguarding

You may be required to undertake safeguarding checks. Shortlisted consultants will be assessed on our organisational values at the interview stage. The successful consultant will be expected to adhere to our safeguarding policy. We encourage you to read and understand our safeguarding policy, the executive summary of which can be found . We have zero tolerance for violence against children, beneficiaries, and staff.

Equal Opportunities

Girl Effect Services is committed to equal opportunity regardless of race, colour, ancestry, religion, sex, national origin, sexual orientation, age, marital status, disability, gender, gender identity, or expression. We are proud to be an equal-opportunity workplace.

We are committed to building an increasingly representative organisation that works extensively with the communities we serve. To this end, due regard will be paid to procuring consultancy service organisations and individuals with diverse professional, academic and cultural backgrounds.