Final Evaluation of PSPU Project 2023-2026 – Ukraine 130 views

Terms of Reference (TOR) for the Final Evaluation of PSPU Project 2023-2026

(To view the TOR in a Word document: Final Evaluation_PSPU_TOR.docx)

Summary

Purpose:

The purpose of this Final Evaluation is to assess the relevance, efficiency, effectiveness, and coordination of the PSPU Project, to inform the continued response in Ukraine, as well as to provide lessons and recommendations for the PSPU Consortium.

Commissioner(s):

Jakob Harbo, Country Manager of the Danish Red Cross in Ukraine

Audience:

The primary audience for this Final Evaluation would be the Danish Red Cross, Danish Refugee Council, Dignity, Ukrainian Red Cross Society, and other local Partners of the PSPU Consortium. The secondary audience would be donors and the government, as well as other local and international humanitarian actors.

Duration of consultancy:

65 working days

Estimated dates of consultancy:

February– May 2026

Geographical scope of consultancy:

Kyiv and Lviv

Reporting to:

Emergency Management Team

1. Background

The armed conflict in Eastern Ukraine rapidly escalated on the 24th of February 2022 and expanded to the whole territory of Ukraine with temporary occupation of regions in the North, East and South. Nearly four years after the large-scale escalation, the most severe fighting is taking place in the East and South with massive airstrikes damaging the energy infrastructure across the country, but the situation remains volatile with likelihood for both escalation and de-escalation in certain areas. From February 2022, the conflict has claimed more than 6,374 civilian lives2 and continues to impact millions of Ukrainians. Up to 7 million people were forced to leave their homes in conflict affected areas and more are expected to move to other regions of Ukraine. The protracted nature of this crisis has seriously affected the regional economy, the environment and overall social cohesion in the government-controlled areas of adding a significant humanitarian burden on state and public institutions, communities, and civil society. The deteriorated situation has gradually barred access of conflict affected people and vulnerable groups to basic services, participation in community life and labour market, and challenging the overall recovery process in affected regions.

Humanitarian actors and authorities have been addressing the needs, but the assistance and services provided remain insufficient compared to the extensive needs of the conflict-affected population. Through this project, the Danish Red Cross, in collaboration with experienced partners in relevant sectors, aimed to enhance the timeliness and quality of the humanitarian response, to complement the effort of other actors responding in Kyiv and Lviv region in the following areas:

• Mental health and psychosocial support as well as legal assistance by extending and strengthening the referral network of civil society and public service providers. Pats figures indicated that around 32% of the IDPs in Ukraine4, and out of those women IDPs, suffered from mental disorders including PTSD, depression, and anxiety. These problems have been exacerbated since the conflict escalation. MHPSS challenges and identified needs have mirrored the general increase of humanitarian needs.
• Vulnerable groups, including survivors of SGBV, required special attention and priority. These groups suffer not only from the negative psychological and physical effects of traumatic events that reduce their well-being and hamper their functioning; they are also often marginalised in their communities due to stigma that challenges their resocialization.
• Youth development, the project is supporting the development of resilient and active youth (age 14-35) and supported them with transitioning during the challenging period of their lives following the past years of COVID-19 lock-down and restrictions, and with almost four years of large-scale armed conflict that has led to displacement, while going through personal challenges related to education, employment, and family building. The support is based around interactive life skills programmes and through building synergies with local implementers of the law “On Basic Principles of Youth policy” effective since May 2021 strengthening the institutional capacity of youth and children’s associations.
• International Humanitarian Law, to address the concern around the domestic implementation of IHL in Ukraine. It was found that the Ukrainian legal measures incorporate some, but not all, of the main IHL requirements. Although the report assessed that there was in general a good commitment by the Government of Ukraine towards greater implementation of IHL, it also identifies certain gaps and inconsistencies. Some of the gaps and inconsistencies mentioned in the report relates to the framework for the prosecution of war crimes and serious violations of IHL, fundamental and judicial guarantees to protected persons incl. e.g., POWs and civilian medical personnel, steps to safeguard cultural property and lack of prohibition of anti-personnel mines or regulations on explosive remnants of war. In addition, reports identifies that there were gaps and inconsistencies in terms of the Ukrainian government implementation of its IHL obligations regarding dissemination, promotion, and training on IHL, across five identified key target groups for this obligation. Namely, the armed forces, police and security forces, university students, public officials, and the general public10.

2. Purpose and Scope

The purpose of this Final Evaluation is to assess the relevance, efficiency, effectiveness, and coordination of the PSPU Project, to inform the continued response in Ukraine, as well as to provide lessons and recommendations for the PSPU Consortium.

The Final Evaluation will focus on the PSPU project including the collaboration within-country participating Consortium Partners, and other humanitarian partners and local authorities as appropriate from the beginning of project implementation until the time the evaluators collect data.

• Relevance would include the extent the PSPU project has been targeted to meet the needs of the most vulnerable
• Efficiency and Effectiveness would reflect on successes and challenges faced by the Consortium Partners and their local partners in ensuring effective and efficient delivery of services to those in need
• Coordination would reflect on successes and challenges faced by the Consortium Partners in internal coordination of the PSPU project along with

In addition to Consortium Partner stakeholders and the target population of assistance and services, the Final Evaluation will also consult with key external stakeholders including partner organizations and government agencies relevant to the review objectives. Geographically, the review will encompass Kyiv and Lviv oblasts served by the PSPU Consortium as part of the PSPU Project.

3. Evaluation Criteria and Key Questions

The review criteria and key questions to be addressed in this Final Evaluation are listed below as guidance and are expected to be further fine-tuned by the Final Evaluation team in consultation with the Evaluation Management Team and relevant stakeholders.

1. Relevance

• Is the assistance provided by the Consortium relevant to the needs of the affected population, including both women and men and the target groups, and is of a quality and scale that meets Red Cross Red Crescent commitments and expectations?
• To what extent were the interventions and targeting of beneficiaries based on assessed and expressed needs, risks, vulnerabilities and capacities of the communities, including those who are the most vulnerable or marginalized?
• Whether the PSPU reports and progress towards targets were used as guidance for where to prioritize time and resources, exploring the relevance of the indicators with low progress to date?
• Is the intervention consistent with the interventions of other actors in the same context, such as supporting complementarity, harmonization, and co-ordination with government and other relevant service providers?

1. Effectiveness

• To what extent is an effective management structure in place within URCS, and DRC in providing direction, clarity, and well-communicated decision-making for the operation and the prioritization of interventions?
• How well are the monitoring and learning processes along with information management systems used in the response and can they be improved?
• Did the program/project achieve its intended outcomes? What are the intended or unintended effects of the programme, either positive or negative, direct or indirect?
• Were there any unintended outcomes?
• Are the objectives of the program/project being achieved?

1. Efficiency

• Were objectives achieved on time? (and budget)
• Were activities cost-efficient? (What was the cost of delivering outputs? How were cost drivers managed?)

1. Coordination

• How effective were the different coordination groups (Management, finance, PMEAL, sectoral groups, etc) in ensuring cooperation and collaboration?
• What strengths and weaknesses did a consortium design bring to meeting the objectives. What should be replicated in the future and what can be improved upon for future consortium designs?

4. Review Methodology and Process

The Final Evaluation will be managed according to the IFRC Guide and Procedures. The methodology will adhere to the IFRC Framework for Evaluations, with particular attention to the processes upholding the standards of how evaluations should be planned, managed, conducted, and utilized. For this evaluation, an Evaluation Management Team (EMT) will be established to review the findings and endorse it. The EMT will have members from Danish Red Cross, Danish Refugee Council, and Dignity. The EMT will be entrusted to guide the evaluation process, including by providing strategic inputs across the whole process, from the design phase to the delivery and comment on the final report.

Interviewees will include Danish Red Cross, Danish Refugee Council, Dignity, URCS, and other local partner personnel, local government authorities, beneficiaries and potentially, other non-movement donors and partners (Danish NGOs, WFP, UNICEF, GIZ, Save the Children, etc.).

The specific evaluation methodology will be detailed in close consultation between the EMT, Commissioner, and the Consultant, but will draw upon the following primary methods:

• Desk review of operation background documents, relevant organizational background and history, including Consortium National policies and SOPs, prior Consortium reports, and any relevant sources of secondary data, such as Post Distribution Monitoring (PDM), learning review report, and other surveys and report from Consortium and participants in the operation.
• Field visits/observations Selected locations of the response particularly those areas of multi-sectoral and diversity of the response (complex vs simple).
• Key informant interviews (institutional and beneficiaries as appropriate).
• Focus group discussions (institutional and beneficiaries) as time and capacity allow.

The evaluation consultant / team is responsible for refining the data collection methodology and plans, as well as proposing relevant approaches, frameworks and/or methods for data analysis.

The successful evaluation team will develop an inception report after the initial desk review for submission and approval to the EMT. This inception report should outline a detailed proposed methodology that must consider: Sampling method is to be recommended by the evaluator, as long the final sample to be evaluated on includes both Consortium Partners involved in the operation interventions, and the ‘most vulnerable’ beneficiaries.

Data collection methods and pace are to be decided by the evaluator, in consultation with the Evaluation Management Team but should consider the reality of difficult-to-reach areas. The evaluation consultant should visit a representative number of communities in the operational areas. This will be agreed with Evaluation Management Team (EMT) based on resources available.

Data quality assurance

The evaluation consultant (company/team) will ensure data quality control measures and document their protocols, in consultations with the evaluation management team.

5. Roles & Responsibilities

Consultant(s)

• Development of inception report with detailed methodology and data collection tools and final evaluation in Ukrainian and English.
• Responsible for collection of data, following up on appointments with support from Consortium Partners.
• Perform data quality control and maintain data quality throughout the process.
• Submission of invoices and necessary documentation in time to facilitate payment as agreed in the contract.
• Handover of de-identified raw data and signed consent forms to Danish Red Cross upon completion of assignment.

Danish Red Cross

• Support with coordination of tasks between all parties – Danish Refugee Council, Dignity, URCS and the consultant.
• Provide all the necessary secondary information for desk review that Danish Red Cross owns
• DRC and URCS will jointly select a most suitable consultant.
• Review of deliverables including the inception report, data collection tools, draft and final positioning study paper.
• Responsible for preparation and execution of contract with selected consultant.
• Facilitate payment of consultant in accordance with the payment schedule stipulated in the contract.
• Support consultants with logistical planning, providing information necessary for data collection.

Danish Refugee Council

• Provide all the necessary information for desk review that Danish Refugee Council owns
• Support consultants with logistical planning, providing information necessary for data collections
• Review of deliverables including the inception report, data collection tools, draft and final evaluation
• Provide access and be the link between the community, stakeholders and the consultant and coordinate data collection activities (identifying respondents together with consultants and setting up appointments from relevant Danish Refugee Council programming)

Dignity

• Provide all the necessary secondary information for desk review that Dignity owns
• Support consultants with logistical planning, providing information necessary for data collections
• Review of deliverables including the inception report, data collection tools, draft and final evaluation
• Provide access and be the link between the community, stakeholders and the consultant and coordinate data collection activities (identifying respondents together with consultants and setting up appointments from relevant Dignity programming)

URCS HQ (Director of Programmes, Deputy Director of Programmes, Quality Control and Accountability Unit, Research & Analysis unit)

• Review of deliverables including the inception report, data collection tools, draft and final positioning study paper.
• Support consultants with logistical planning, providing information necessary for data collection.
• Provide all the necessary secondary information for desk review.
• Provide access and be the link between the community, stakeholders and the consultant and coordinate data collection activities (identifying respondents together with consultants and setting up appointments).
• Responsible for coordination of tasks between parties within URCS and the consultant.
• URCS and DRC will jointly select a most suitable consultant.
• URCS and DRC will have exclusive rights to all data generated from the assignment.

URCS (Regional organizations and local branches)

• Provide necessary information from local and regional level on URCS programming
• Act as the link between the community, stakeholders and the consultant at the local and regional levels and coordinate data collection activities (identifying respondents together with consultants and setting up appointments from relevant URCS programming)
• Participate in relevant KIIs and FGDs

6. Review Deliverables and Illustrative Timeline

The evaluation consultant is expected to deliver the following outputs:

• Inception Report: The inception report will be a scoping exercise for the review/evaluation and will include the proposed methodologies, data collection and data quality control measures, as well as reporting plans with draft data collection tools such as interview guides, the allocation of roles and responsibilities within the team, a timeframe with firm dates for deliverables, and the travel and logistical arrangements for the team.
• Debriefings / feedback to management at all levels: The team will report its preliminary findings to the EMT prior to leaving the country to validate the findings in a participative manner. This will include a presentation to key stakeholders as determined by the EMT, a detailed PowerPoint presentation and sufficient time for discussion and suggested edits. This workshop will be attended by representatives from branches, sectoral leads, regional and interregional staff.
• Draft report: A draft report identifying key findings based on facts and will separate from the report opinions or rumours, conclusions, recommendations, and lessons for the current and future operation, will be submitted by the evaluation consultant within a week after presenting the initial findings.
• Final report: The final report will contain a short executive summary (no more than 1,000 words) and a main body of the report (no more than 10,000 words) covering the background of the intervention evaluated, a description of the evaluation methods and limitations, findings, conclusions, lessons learned and clear recommendations. Recommendations should be specific and feasible. The report should also contain appropriate appendices, including a copy of the ToR, cited resources or bibliography, a list of those interviewed and any other relevant materials. The final report will be submitted one week after receipt of the consolidated feedback from the EMT. Details of the final report are outlined in the table below.

Suggested final report outline:

No.

Content

Description

01.

Executive Summary

Summarizes the overall findings of the evaluation with key conclusions and not more than 10 key recommendations. Executive Summary must be specific to the evaluation and clearly outline the specific context of the interventions.

02.

Background

Outlines the overall objectives, aims, intervention strategy, policy frameworks, targets, main stakeholders, financial frameworks, institutional arrangements.

03

Methodology

Outlines the overall approach used and the rationale why the approach used, the tools applied and the key assumptions. It will focus on consideration for relevance and, coverage, efficiency / effectiveness / in function of the internal and external issues. Any limitations the evaluation faced should be listed here.

04.

Findings

Outlines the findings of the evaluation based on the SADDD criteria.

05.

Conclusions

Outlines the main conclusions that have emerged from the findings

06.

Lessons learned and recommendations

Provides general overall recommendations, including on cross-cutting issues that affect not only the PSPU programmes and operations but also on relevant strategies, advocacy, and coordination.

All products arising from this evaluation will be owned by Danish Red Cross. The evaluator(s) will not be allowed, without prior authorization in writing, to present any of the analytical results as his/her own work or to make use of the evaluation results for private publication purposes.

The draft and final reports will be submitted through the EMT, who will ensure the quality of the report, providing input if necessary. The EMT will submit the report to the key stakeholders interviewed for any clarifications. The Commissioner will oversee a management response and will ensure subsequent follow-up.

The following is an illustrative timeline that will be revisited and refined with more details during the inception stage of the Final Evaluation:

Time Schedule*

Activities

Deliverables

Week 1

(Starting February 16th)

• Desk review of background information
• Initial briefings and meetings to inform the development of an inception report
• Knowledge gained on PSPU project

Week 2

• Development of detailed inception report, data collection/analysis plan and schedule, draft methodology, and data collection tools
• Inception report with detailed data collection and analysis plan, methodology, and data collection tools

(Due February 27th)

Weeks 3-4

• Review of Inception Report and data collection tools
• N/A

Weeks 4-9

• Data collection and updating
• Validation workshop
• End-of-week progress reports
• Validation workshop

Week 10

• Prepare first draft of the PSPU Evaluation
• First draft of the PSPU Evaluation recommendations and program options

(Due April 30th)

Week 11-12

• Review by PSPU Consortium Partners
• N/A

Week 13

(Ending May 22nd)

• Revise and submit the final report.
• Final presentation on the main findings and recommendations
• Final Evaluation Report
• Final virtual presentation (PowerPoint or similar)

(Due May 22nd)

*Tentative timetable based on date contract is signed

7. Evaluation Quality and Ethical Standards

The evaluator(s) should take all reasonable steps to ensure that the review/evaluation is designed and conducted to respect and protect the rights and welfare of people and the communities of which they are members and to ensure that the evaluation is technically accurate, reliable, and legitimate, conducted transparently and impartially, and contributes to organizational learning and accountability. Therefore, the evaluator(s) should adhere to the evaluation standards and specific, applicable processes outlined in the IFRC Framework for Evaluation. The IFRC Evaluation standards are:

The IFRC Evaluation Standards are:

1. Utility: Evaluations must be useful and used.
2. Feasibility: Evaluations must be realistic, diplomatic, and managed in a sensible, cost-effective manner.
3. Ethics & Legality: Evaluations must be conducted in an ethical and legal manner, with regard for the welfare of those involved in and affected by the evaluation.
4. Impartiality & Independence: Evaluations should be impartial, providing a comprehensive and unbiased assessment that considers the views of all stakeholders.
5. Transparency: Evaluation activities should reflect an attitude of openness and transparency.
6. Accuracy: Evaluations should be technically accurate, providing sufficient information about the data collection, analysis, and interpretation methods so that its worth or merit can be determined.
7. Participation: Stakeholders should be consulted and meaningfully involved in the evaluation process when feasible and appropriate.
8. Collaboration: Collaboration between key operating partners in the evaluation process improves the legitimacy and utility of the evaluation.

It is also expected that the evaluation will respect the seven Fundamental Principles of the Red Cross and Red Crescent: 1) humanity, 2) impartiality, 3) neutrality, 4) independence, 5) voluntary service, 6) unity, and 7) universality.

8. Evaluator And Qualifications

Evaluator(s), with knowledge of local context who will provide an independent, objective perspective as well as technical experience on evaluations, will be hired to conduct the review/evaluation. The evaluator will be the primary author of the evaluation report. S/he will not have been involved or have a vested interest in the URCS operation or context being evaluated, and will be hired through a transparent recruitment process, based on professional experience, competence and ethics and integrity for this evaluation. The evaluator will report on progress or challenges to the evaluation management team. The evaluator should have the following characteristics:

• Demonstrable experience in leading evaluations of humanitarian programmes responding to conflict and displacement crises and preferably previous experience of conducting evaluation in Ukraine or impacted countries.
• Knowledge of activities generally conducted by humanitarian organizations in the MHPSS, Youth, Livelihoods, and IHL sectors.
• Field experience in the evaluation of humanitarian or development programmes, with prior experience of evaluating Consortium programmes desirable.
• Strong analytical skills and ability to clearly synthesize and present findings, draw practical conclusions, make recommendations and to prepare well-written reports in a timely manner (examples of previous work).
• Previous experience in coordination, design, implementation, and monitoring and evaluation of humanitarian programmes.
• Experience in qualitative data collection and data analysis techniques, especially in emergency and PSPU operations.
• Demonstrated experience in data quality control.
• Knowledge and experience working with the Red Cross Red Crescent Movement preferred.
• Demonstrated capacity to work both independently and as part of a multi-discipline, multi-national team.
• Ability to work within tight deadlines and manage with available resources.
• Fluent in spoken and written English. Advantageous to be fluent in Ukrainian.
• Strong interpersonal skills.
• Relevant degrees or equivalent experience.
• Availability for the time period indicated.

1. Appendices (To Be Presented After Appointment)
2. Package of Reference Documents for PSPU Project
3. Relevant Consortium Partner programmatic documents
4. Stakeholders list

9. Application procedures

Interested applicants should submit their expression of interest by emailing Mathew Vicknair at [email protected], by midnight 10 February 2026 (Ukraine time).

Application materials should be submitted in English, which will include:

1. Curriculum Vitae (CVs) of consultant.
2. Cover letter clearly summarizing your experience as it pertains to this evaluation, daily consultancy fees inclusive of rate for the national consultant (as relevant), and three professional references.

To view the TOR in a Word document, use the link below:

Final Evaluation_PSPU_TOR.docx