FY 16: Evaluating the Effectiveness of Shelter, Health, and Education Programs for Iraqi and Syrian Refugees in Jordan, Lebanon, and Turkey
This evaluation was performed between 11/16/2015 and 03/22/2017, and the report was submitted on 02/22/2017.
Purpose of the Evaluation and Questions Addressed
The Bureau of Population, Refugees, and Migration (PRM) contracted Social Impact (SI) to examine the effectiveness of shelter, health, and education (SHE) programs for Syrian and Iraqi refugees in Jordan, Lebanon, and Turkey. The evaluation focused on programs implemented since fiscal year (FY) 2013 by 11 of PRM’s multilateral and non-governmental organization (NGO) partners, with an emphasis on FY 2014 and 2015 activities. The findings and recommendations will guide PRM’s funding decisions and diplomatic engagements, increase operational efficiency, inform PRM Refugee Coordinators’ monitoring efforts, and enable partners to increase their impact. The primary evaluation questions were:
- Were SHE programs supported by PRM and its partners implemented effectively in accordance with best practices (in terms of refugee satisfaction; targeting of vulnerable groups; alignment with refugee preferences; use by host communities; building on existing capabilities; and sustainability)?
- How successful were rental agreements with landlords in allowing refugees to meet their shelter-related needs? What happened when rental agreements ended, and what are the implications for refugee assistance?
- How could cash assistance programming be more effective, in the context of SHE? Were there instances where this cash was used for other immediate needs?
- How and to what extent was programming coordinated with local governments, local organizations, and civil society?
- How could PRM and its partners improve humanitarian programming and diplomacy based on available evidence?
SI used a qualitative evaluation design that included document review, individual and group interviews with stakeholders, and direct observation. The evaluation teams conducted site visits and data collection at program implementation sites in Jordan, Lebanon, and Turkey. Limitations of the design included selection bias and response bias, which are common in qualitative studies of this nature. In addition, security considerations limited access to some stakeholders in Lebanon and Turkey.
- Overall, PRM-supported SHE programs follow best or emerging good practices:
- The majority of beneficiaries interviewed across all three countries are satisfied with the services provided by PRM partners and reported improvements in their quality of life.
- The programs evaluated are generally accessible to vulnerable groups, although very few beneficiaries are Palestinian or Iraqi refugees. However, PRM supports other programs that target those vulnerable groups.
- Most programs align with refugee preferences, but there is a lack of consistency among partners with regard to soliciting and incorporating beneficiary feedback.
- PRM-supported programs build upon existing structures, enhance existing capacities, and fill gaps to address specific needs of Syrian refugees where government-supported services are unavailable.
- The programs evaluated are largely available to host communities. Actual use by host communities varies by country and service type, but it is typically low.
- Host country policies and abbreviated, unpredictable funding cycles threaten the sustainability of PRM-supported programs.
- Where PRM-supported programs offered rental assistance, refugees received temporary financial relief. Unique logistical challenges accompany each rental assistance modality employed by the programs evaluated. Shelter is a major expense for which many refugees need continued support.
- Due in part to information gaps in humanitarian research (especially on the topic of cash assistance programming), PRM partners lack adequate empirical information on the best modalities for implementing SHE programs.
- PRM partners’ coordination with local governments and local civil society organizations is variable and could improve in most cases.
- Ensure—and make any necessary improvements for—targeting of and support to the most vulnerable Syrian, Iraqi, and other non-Syrian refugees.
- Improve outreach to beneficiaries and vulnerable groups, and involve beneficiaries in program design.
- Conduct program monitoring and research to strengthen the evidence base for SHE programming.
- Continue advocacy, coordination, and information-sharing efforts with central and local host governments, donors, and NGO partners.
- Ensure that host communities are included in programming, when appropriate, to strengthen social cohesion and integration.
- Increase consultation and implementation of activities in coordination with local governments and civil society actors to promote ownership and sustainability.
- Develop sustainability plans for SHE interventions, and foster long-term integration into existing structures.
Links to Evaluation Reports