Ended Final Evaluation: Consulting Firms & Consultants

The Job

TERMS OF REFERENCE

FINAL EVALUATION OF THE OFDA-FUNDED EMERGENCY LIFE-SAVING AND EARLY RECOVERY ASSISTANCE PROJECT IN YEMEN

 

Background:

The International Rescue Committee (IRC) has been providing life-saving support to conflict affected communities in Yemen since 2012 through delivering health, nutrition, food security, protection, WASH and education interventions in numerous locations. With funding from the Office of U.S. Foreign Disaster Assistance (OFDA), the IRC started implementing a 19 months project in March 2019 to minimize avoidable loss of life and improve well-being of conflict affected Yemenis through emergency response and early recovery activities for more than 200,000 people. While this project includes a continuation of many of the emergency humanitarian activities that the IRC has implemented in Yemen for the past few years, it also includes new interventions within the economic recovery, agricultural and food security sectors to support new livelihoods development and household-level food production. These interventions will contribute to ensuring that targeted beneficiaries have reduced reliance on aid and are empowered to establish their own income opportunities. The project is scheduled to end in September 2020 and aimed at achieving the following objectives;

  • Reduce morbidity and mortality among conflict-affected Yemeni populations through improved access to primary & reproductive health care services.

  • Reduce mortality and morbidity associated with moderate and severe malnutrition of children under 5 years and pregnant and lactating women living in the targeted areas.

  • Ensure that people affected by conflict are protected from water, sanitation and hygiene related diseases.

  • Enable food insecure households have access to available and adequate nutritious food to increase nutritional status.

  • Enable food insecure households meet their basic needs and avoid negative coping strategies.

  • Increase women’s economic empowerment through generating income and strengthening control over economic resources.

  • Ensure that women and children affected by crisis receive timely care and support services

Purpose and scope of Evaluation

The IRC seeks to undertake a final of project evaluation as a means of assessing to what extent set objectives were achieved, including the overall results, effects/changes realized, good practices, and challenges faced and lessons learnt.

The evaluation is planned to take place in September 2020 and the duration to be covered under this evaluation is the entire project implementation period minus one month (March 2019 - August 2020). The geographical scope is the project’s operational districts in the seven governorates of Aden, Lahj, Abyan, Shabwah, Al Dhale, Hodeidah and Sana`a/Amanat Al Asimah. The evaluation will review all interventions including health, nutrition, Environmental Health (EH), Economic Recovery and Development (ERD), and protection. The findings will be used by the IRC, partners and the donor to understand what worked or did not work so as to develop and improve evidence-based interventions aimed at improving the welfare of populations affected by conflict in Yemen.

Specific Objective

The specific objectives of the evaluation are to;

  • Assess the extent to which the project achieved its set objectives and targets

  • Analyse the relevance of the project in improving the welfare of populations affected by conflict in Yemen as perceived by direct beneficiaries

  • To capture intended and unintended changes in behaviours, actions, ownership, capacities, and relationships of program stakeholders.

  • To assess how the project impacted the lives of the target population and contributed to a reduction of their vulnerability

  • To draw key lessons learned from the project and make recommendations that will help inform formulation and design of future projects targeting conflict affected populations in Yemen

  • Examine whether the project was implemented in the most efficient manner 

Evaluation Criteria

The evaluation will be informed by and structured around the OECD/DAC criteria with questions framed to assess the relevance, effectiveness, efficiency and sustainability of actions and achievements.

Proposed Methodology

 Due to ongoing conflict and the prevalence of COVID-19 in, international travel to Yemen is not possible. There are limitations on the ground which restrict large gatherings and in-person meetings; therefore, significant portions of this evaluation will need to be conducted remotely and/or through previously established contacts inside Yemen. The consultant will be expected to utilize creative methodologies and tools in light of COVID-19 challenges.

The evaluation will rely on a range of quantitative and qualitative methods including a survey, remote focus group discussions, key informant interviews, semi-structured interviews, desk reviews, and, where possible, field observations and visits. The evaluation will collect information from various project stakeholders including beneficiaries, community leaders, and local authorities. A more detailed study approach and methodology will be developed by the selected consultancy firm.

 The consultancy firm will undertake the following tasks, among others;

  • Design the end line evaluation plan, including developing a detailed methodology and data collection tools. Incorporate feedback from the IRC as required.

  • Conduct meetings with key IRC program and Monitoring, Evaluation, Accountability & Learning (MEAL) staff to further understand the project and evaluation objectives

  • Undertake a detailed desk review of program documents including available secondary data

  • Train enumerators and lead field data collection, management and analysis

  • Write inception and final reports; incorporate feedback from reviewers

  • Make a presentation on key findings and recommendations to the IRC senior management team for their input into

  • Ensure the safeguarding of project beneficiaries reached by the data collectors, mitigate any ethical concerns and ensure the safe management of beneficiary data

Evaluation questions

The evaluation shall examine project implementation against the hereunder criteria by addressing the following questions:

Criteria

Questions

Relevance

 

  • How relevant was the project in improving the welfare of populations affected by conflict in Yemen?

  • Was the project design relevant to addressing underlying causes of vulnerability? Did the outputs contribute to the achievement of outcomes? Did the outcomes contribute to achieving the overall objective?

  • To what extent was the project aligned to the needs of the target population as well as the local context?

  • How did the IRC ensure that the most vulnerable people, including men, women, boys and girls were selected for support? How were the target groups selected?

  • What feedback channels were mostly used by project clients including women, men, boys and girls including vulnerable groups?

  • Was the feedback of clients considered during project implementation and decision making?

  • What ways did project teams adapt to increase client collaboration and engagement?

Effectiveness

 

  • To what extent were the project objectives achieved?

  • What major factors facilitated or hindered the achievement of project objectives?

  • How was access, utilization and quality of health, nutrition, WASH, livelihoods, agriculture and protection services enhanced and what challenges were faced?

  • To what extent did project interventions contribute to reducing vulnerability among populations affected by conflict in Yemen?

  • What approaches did the project use to mainstream gender, age, disability and what was the impact of the project on gender equity and related issues?

  • What approaches were used to ensure program quality especially during implementation?

  • How effective was the integrated project approach in improving the welfare of beneficiaries?

  • Did the project teams use regular monitoring data in their decision meetings and for project improvement?

  • What improvements were documented on the basis of decision taken on monitoring data?

Efficiency

 

  • Was the project implemented in the most efficient manner and with the best use of existing resources? What key limitations were faced on this front?

  • How cost-effective was the project? What cost-effective alternatives could have been used?

  • Were project outputs and targets achieved on time? And if not, what were the key challenges?

 

Sustainability

 

  • What exit strategy options were put in place to ensure that the end of the project does not negatively affect the welfare of beneficiaries?

  • How has the IRC supported the government and other partners to develop capacities and mechanisms to ensure ownership and scale up of interventions? 

 

Coordination

 

  • How did the project take advantage of existing coordination mechanisms and complementarities/synergies with other partners in the field to avoid gaps, duplication and conflicts?

  • What gaps were identified and addressed by partners in the field?

 

Impact

 

  • What were the intended and unintended positive and negative impacts of the project as perceived by beneficiaries?

 

Lessons learned and replicability

  • What are some of the key lessons learned as a result of this project that can be shared and replicated?

  • How was data, information, learning or knowledge of the project managed during the project life?

 

Note: Age, disability and gender disaggregation must be applied as much as possible for each of the evaluation criteria.

Key deliverables

The hired firm will be expected to submit the following;

Deliverables

  • A detailed inception report and data collection tools after signing the consultancy agreement

  • A draft report for review by the IRC Yemen senior management team, program and MEAL coordinators

  • A PowerPoint presentation covering the study background, objectives, methodology, key findings and recommendations

  • A final report after incorporating comments from the IRC Yemen team

The draft and final report shall have the following structure:

  1. Executive Summary

  2. Introduction

  3. Methodology, including sampling frame

  4. Limitations of the evaluation

  5. Analysis and findings of the study, both quantitative and qualitative. Should specify how the qualitative data was analyzed and also how the data compared to desk reviews.

  6. Evidence of success/failures

  7. Conclusions, recommendations, lessons learned and best practices

  8. Annexes

  1. Relevant maps and photographs of the study areas

  2. Bibliography of consulted secondary sources

  3. Finalized data collection tools (in English and Arabic)

  4. List of key informants

  5. A clean dataset including all interview transcripts and recordings (both quantitative and qualitative) in agreed form.

 

Qualifications and Experience of the Lead Consultant

  • Fluency in English and Arabic is required

  • An established network and data collection capacities inside Yemen required

  • Experience in leading remote evaluations required, experience in other countries in the Middle East preferred

  • An advanced university degree in a research-related field

  • At least 6 years of relevant experience in leading and managing end line evaluations in the humanitarian or development sectors

  • Technical expertise in complex project evaluation, preferably of integrated programs in emergency contexts

  • Previous experience conducting end line evaluations for large-scale integrated projects

  • Knowledge of tools for mobile data collection (Mainly KoBo, CommCare)

  • Strong skills in the collection and analysis of quantitative and qualitative data

  • Strong interpersonal communication skills

  • Experience with gender sensitive programming and knowledge of gender-sensitive evaluation methods

  • Excellent report writing skills

Management of the Evaluation

The evaluation will be directly implemented and managed by the selected consultancy firm in collaboration with the IRC Yemen. The MEAL Coordinator, with technical support and guidance from the Senior Grants & Accountability Coordinator and Deputy Director of Programs, will monitor its implementation and provide quality assurance throughout the process.

Duration of the Assignment

The evaluation is expected to last 22 working days from the date of signing the consultancy contract. The final report is expected not later than September 30, 2020. Below is a breakdown of the allocated timeline;

Key Tasks

Allocated days

Development of the inception report and data collection tools

4 days

Training of research assistants

2 days

Data collection phase

8 days

Data analysis and report writing

6 days

Revision of the draft report and submission of the final version

2 days

 

To view all the details, please download the attached TORs file below this ad or visit the following link:

https://rescue.csod.com/ux/ats/careersite/1/home/requisition/9969?c=rescue

 

 
How to Apply

Application Process

Qualified firms are requested to submit technical and financial proposals. The proposals must include, but are not limited to, the following items:

  1. A corporate profile highlighting the bidder’s qualifications and relevant experience

  2. An outline of how this evaluation will be completed remotely and / or using existing contacts in target locations within Yemen.

  3. A description of how the bidder will address the evaluation objectives and evaluation criteria.

  4. A detailed methodology including sampling approaches, a work plan and any suggestions to improve the outcomes of the assignment.

  5. Include at least two reports from similar evaluations that were accomplished by the bidder in the last three years with particular emphasis on projects of similar scope and effort. It is desirable that a summary of the past projects be included in the proposal. These reports or work products may be attached as an appendix to the proposal. Please include the following information:

  • Name of client

  • Title of the project

  • Year and duration of the project, including timelines between intermediate steps, such as time duration between RFP and proposal, proposal and project initiation, project initiation and first milestone, etc.

  • References / contact details (emails and telephone numbers) of organizations where the firm conducted similar evaluations

Description of Personnel

For ease of reference, IRC has defined the following categories of consultants/personnel. Bidders are free to provide alternative titles/descriptions for each of the designations used below:

 

 

Role

Years   of Experience

Number of Personnel

Names with Submitted Resumes

1

Lead Consultant

10+

 

 

2

Consultants

8+

 

 

3

Junior Consultants

2-5 years

 

 

Evaluation Process

Each proposal will be assessed first on its technical merits and subsequently on its price. The proposal obtaining the overall highest score after adding the scores for the technical and financial proposals is the proposal that offers best value for money and will be selected for the contract.

The proposals will be evaluated against the following elements:

A: Technical Proposal

The total amount of points allocated for the technical component is 70. IRC evaluators will read the submission and give scores according to the table below. Only bidders that obtain 50 points and above from the technical evaluation will be considered for the financial evaluation stage.

Technical Evaluation Criteria (70%)

Max. Points

1. Overall Response

10

1.1

Completeness of the technical proposal with reference to requirements outlined in the Application Process section above

5

1.2

Quality and completeness of the proposed work plan to achieve the evaluation objectives

5

2. Overall Experience of the Firm and Key Personnel

40

2.1

Relevant experience in leading remote and large-scale evaluations for integrated projects in Yemen or other humanitarian contexts

10

2.2

Quality of evaluation reports from previous engagements

10

2.3

Capacity to provide sufficient team members in Yemen for the scale and scope of work required

10

2.4

Recommendations from previous clients preferably for evaluation assignments of a similar scope

10

3. Proposed Methodology and Approach

20

3.1

Detailed methodology that aligns with evaluation objectives and the OECD/DAC evaluation criteria

20

 

 

 

 

B. Financial Proposal

This should be filled out by the bidder to include the full list of daily rates for personnel. The rates should be fully inclusive, considering communications, overhead, domestic travel for data collectors inside Yemen, etc. Due to COVID-19, international travel will not be possible. The total amount of points allocated for the price component is 30 and will be calculated as weighted score inversely proportional.

All prices/rates quoted must be exclusive of all taxes as IRC is a tax-exempt organization.

 

To apply for this consuultancy, please visit the following link:

https://rescue.csod.com/ux/ats/careersite/1/home/requisition/9969?c=rescue

 

 

Files
  • Employer: IRC
  • Category: Consultant
  • Location: Yemen
  • Posted on: 13 August 2020
  • Deadline: 20 August 2020

TOP