FY 16: Evaluating the Effectiveness of PRM Multilateral Partners
The U.S. Department of State (DoS) Bureau of Population, Refugees, and Migration (PRM) contracted Social Impact, Inc. (SI) to conduct a performance evaluation to examine the effectiveness of PRM’s multilateral partners (international organizations, or IOs) in assisting internally displaced persons (IDPs) in Ukraine and preparing for the eventual transition from relief to development. The evaluated IOs include the United Nations High Commissioner for Refugees (UNHCR), the International Committee of the Red Cross (ICRC), the International Organization for Migration (IOM), and the United Nations Population Fund (UNFPA). PRM’s Europe, Central Asia, and the Americas and Policy and Resource Planning offices will use the evaluation findings and recommendations to guide their programmatic and diplomatic decision making in planning for longer-term development regarding the local integration of IDPs in Ukraine.
Purpose of the Evaluation and Questions Addressed
The evaluation answers the following questions:
- Access to services: What are the on-the-ground realities for an IDP who seeks to obtain IDP registration documentation, a job, education, healthcare, a lease, a propiska, social benefits (i.e., pensions), legal assistance, and the right to vote in his/her new community? What legislative or policy changes are needed to improve access?
- Assistance: Have PRM’s multilateral partners been successful in meeting the humanitarian needs of Ukrainian IDPs? Will assistance provided to date support local integration over the short, medium, and long term?
- Beneficiary selection: What are current processes by government entities/UN agencies/non-governmental organizations (NGOs) for selecting beneficiaries for assistance? Are there ways to ensure that the most vulnerable are prioritized?
- Beneficiary feedback: To what extent did IDPs report receiving integration assistance from UNHCR, ICRC, and IOM? Did they feel that assistance received was helpful or, if not, what forms of assistance would have been preferred?
- Best practices: Do PRM’s partners make use of best practices in their programming and engagement? Have there been any unintended consequences?
Data collection methods aimed to generate the highest-quality and most credible evidence, taking into consideration time, budget, and other limiting factors. The evaluation team (ET) employed document review, site observations, key informant interviews (KIIs), and group and individual interviews with beneficiaries to collect both primary and secondary data, the majority of which was collected in Kyiv, Kharkiv, Dnipro, Zaporizhzhia, Kherson, and Lviv Oblasts. Key informants included multilateral partners’ staff, international and local NGO implementing partners (IPs), project beneficiaries, key stakeholders, PRM, representatives of other donors, and external actors.
Selection of Key Findings
Evaluation Question 1 - Access to Services
- Housing is a top priority need for nearly all interviewed IDPs. The ET found that there are chronic housing needs across the whole country, which have been amplified by displacement, with an acute shortage of adequate accommodation in the areas with the highest influx of IDPs.
- For the most part, IDPs do not obtain permanent residence registration, called propiska, because they are unable to get the required documentation. A propiska is directly connected to eligibility to access state subsidies, jobs, and the right to vote in local elections.
- There is a long list of issues that prevent IDPs to register for and receive social benefits from the government. Chief among them are the government’s slow processing speed, strict IDP-verification requirements, and the choice to use only Oschadbank to distribute pensions.
Evaluation Question 2 - Assistance
- Interviews with external actors revealed a need for IOs implementing indirectly in NGCAs to identify a more effective and efficient approach in DPR and LPR.
- IOs collaborated well amongst themselves as well as with their implementing partners and local authorities to successfully design interventions grounded in assessments/consultations with intended target beneficiaries and local government officials.
Evaluation Question 3 - Beneficiary Selection
- IDPs living in rural areas are not well targeted and reached by humanitarian organizations as reported by beneficiaries in Kherson, Kharkiv, and Lviv.
- Despite IO’s efforts to define and target the most vulnerable, beneficiaries reported that the lives of those IDPs who were more or less stable are rapidly becoming vulnerable
Evaluation Question 4 - Beneficiary Feedback
- Lack of acceptance of IDPs and negative attitudes towards IDPS among host communities is a factor that negatively affects integration of IDPs in new communities.
- Findings suggest the following preferred forms of assistance: sustainable housing, access to medication, psychological support, improved living/shelter conditions, and access to microcredit/loan and income-generating opportunities.
Evaluation Question 5 - Best Practices
- The evaluation findings indicate that most IOs, fully or partially, utilize good practices in their activities and engagement.
- Introduce/amend legislation and policies to improve the quality of, and IDPs’ access to, services.
- Adjust programs and activities to improve effectiveness of assistance.
- Engage in information sharing and outreach, both with the wider international community and with government structures and beneficiaries.
- Partners should continue to be actively engaged in collaboration and partnership-building with national and local authorities, community-based organizations (CBOs), and other NGOs. PRM should support and encourage this.
- Ensure that host communities are included in programming to strengthen IDP integration.
- Develop a strategy to manage the aftermath of displacement and mainstream IDP issues into local development plans and initiatives.
- Engage development actors to identify potential humanitarian-development linkages and plan for IDP integration.
Additional information on PRM’s use of the evaluation recommendations can be found in the PRM Multilateral Partners in Ukraine Evaluation Action Plan.
Links to Evaluation Reports