Evaluation and impact assessment

 

 

The WCC (https://www.oikoumene.org/) is an ecumenical fellowship of churches founded in 1948. It brings together 352 member churches from all over the world, including Orthodox, Anglican, Baptist, Lutheran and Reformed denominations, to promote Christian unity in faith, witness and service for a just and peaceful world. 

 

The EAPPI (https://www.oikoumene.org/what-we-do/eappi) is a collaborative ecumenical and multi-faith initiative founded by the WCC in 2002. Its primary aim is to bring to an end systemic human rights violations under the framework of International Humanitarian Law (IHL) and International Human Rights Law (IHRL) related to the Occupation in the West Bank and East Jerusalem. The programme welcomes participants from WCC member churches, ecumenical partners, and members from the Jewish and Muslim faiths, as well as from various civil society organisations. These participants known as Ecumenical Accompaniers (EAs) live and work with communities at risk under Israeli occupation for a period of three months. During this time, they monitor and report on violations of IHL and IHRL offering a protective presence through their solidarity with these communities. The EAs also gain valuable insights and learn from their presence in Palestine and Israel, taking back their experiences to share with their own communities. 

 

As EAPPI marks its 20th anniversary, the conclusion of its first strategic plan (2017-2021), and after almost two years of absence of EAs on the ground due to COVID-19 (2021-2022), there is a need to evaluate the programme’s progress in achieving its goals. The evaluation will also assess the new context and circumstances including the recent installation of an extreme far-right government with policies favouring settlers in the area, and the ongoing war in Gaza. It will also collect learnings that will help EAPPI and its stakeholders in developing the next strategic plan.

 

The WCC is looking for a contractor to undertake a programme evaluation and produce a comprehensive report that includes a summative evaluation. The contractor could undertake this assignment through one or several consultants. Given the nature of the programme, the consultancy is highly confidential. Materials developed will be owned by the WCC and circulation will be limited. The contractor should abide to strict confidentiality requirements.

 

 

The overall purpose of the evaluation is to assess the results and impact of the EAPPI during the 2017 – 2023 period on the lives of the people it seeks to protect. This evaluation will consider the various measures implemented in response to an organizational capacity assessment conducted by an external consultant in 2018 and evaluate how these measures have affected the programme’s effectiveness and efficiency.

 

Additionally, the evaluation will consider the impact of the COVID-19 pandemic on both the implementation of the programme and the situation of the communities on the ground. Due to the pandemic, it was not possible to deploy Ecumenical Accompaniers (EAs) to the region during 2020-2021. Instead, the programme relied on local staff and partners to continue some accompaniment activities, with an emphasis on advocacy initiatives, particularly online.

 

The findings from this evaluation will assist EAPPI and its stakeholders in formulating the next Strategic Plan by taking into account lessons learned to maximize its impact on vulnerable communities. This impact includes both the short-term protective presence of the EAs and the long-term goal of ending the occupation and achieving a just peace.

 

The evaluation should examine how EAPPI adapted to these challenges and identify any lessons learned.

 

 

The main objectives of this consultancy are to:

a. Assess Programme Impact:

- Evaluate the impact of EAPPI on the lives of the people it seeks to protect during the 2017-2022 period.

- Measure the extent to which EAPPI has contributed to the short-term protective presence and the long-term goal of ending the occupation and achieving a just peace.

b. Evaluate Implementation and Effectiveness:

- Assess the effectiveness of the various measures implemented in response to the 2018 organizational capacity assessment.

- Analyse how these measures have influenced the programme’s overall effectiveness and efficiency.

c. Examine COVID-19 Adaptations:

- Evaluate the impact of the COVID-19 pandemic on the implementation of the programme and the situation of the communities on the ground.

- Assess how the programme adapted to the inability to deploy EAs during 2020-2021 and the reliance on local staff and partners.

- Review the effectiveness of the advocacy initiatives, especially those conducted online, during this period.

d. Identify Lessons Learned:

- Identify key lessons learned from the programme’s adaptations to challenges, including the pandemic and organizational capacity assessment.

- Provide recommendations on how these lessons can be applied to improve future programme implementation and strategic planning.

e. Inform Future Strategic Planning:

- Offer actionable insights and recommendations to EAPPI and its stakeholders for the formulation of the next Strategic Plan.

- Ensure the findings contribute to maximizing the programme's impact on vulnerable communities.

f. Assess Stakeholder Satisfaction:

- Evaluate the satisfaction levels of the programme beneficiaries, partners, and other stakeholders.

- Determine how stakeholder feedback has been integrated into programme improvements.

 

 

The consultant will conduct a comprehensive evaluation of the EAPPI with a multi-faceted approach, focusing on the following key stakeholder groups:

a. Rights-holders: Individuals and communities in the West Bank, including East Jerusalem, to whom EAPPI aims to provide its protective presence.

b. Local Stakeholders: This includes local churches and other relevant groups in Palestine and Israel.

c. National Coordinators/Sending Organizations and Ecumenical Accompaniers (EAs).

d. The World Council of Churches (WCC).

 

A detailed overview of the specific stakeholders targeted in the evaluation will be provided to the selected consultant.

 

The evaluation will be overseen by a steering committee appointed by the WCC, comprising WCC staff, representatives of local stakeholders, and members from the National Coordinators.

 

Evaluation Design and Data Collection:

• The methodology should integrate both quantitative and qualitative approaches, including surveys, interviews, focus groups, and case studies.

• The approach should demonstrate gender sensitivity and conflict sensitivity in all aspects of the evaluation.

• The evaluation is expected to be a learning process, enabling stakeholders to re-examine methods and approaches in light of current circumstances.

• The evaluation should focus on both short-term protective impacts and long-term outcomes, such as policy changes or shifts in community resilience.

• The results will include a qualitative and quantitative impact assessment.

 

Review of Programme Documentation:

• Conduct a review of archived materials related to the overall programme, including background material used in project preparation, former evaluations, approved project documents, project monitoring documents, placement logs, disbursement reports, progress reports, action plans, and other relevant information.

 

Field Visits and Interviews:

• Develop and propose specific methods for data collection, including surveys and interview protocols.

• Outline strategies for triangulating data from different sources to ensure validity and reliability.

• Conduct visits and interviews (in person or by phone) with the following stakeholders:

 o Relevant local stakeholders.

 o EAs involved during the implementation of the project activities.

 o International Reference Group members.

 o Key EAPPI staff in Jerusalem and Geneva.

 o Selected National Coordinators.

 

Data Collection for Desk Study:

• EAPPI strategic plan and reports covering the years 2017-2022.

• Data from EAPPI’s monitoring systems.

• Previous evaluation reports (2004 evaluation by Dan Church Aid; 2005 evaluation by the WCC; 2007 evaluation for the Swedish National Coordination; 2008 impact evaluation; 2012 evaluation by Swedepeace; 2016 Evaluation and 2022 Assessment by MiMAT).

• UNOCHA overviews, plans, reports, and fact sheets, including Protection of Civilians (weekly), The Humanitarian Bulletin (monthly), 2016 Humanitarian Needs Overview, and 2016 Humanitarian Response Plan.

 

The consultant is invited to propose a tailored methodology in an inception report, including actionable recommendations for future programme improvements.

 

 

The contractor is expected to deliver the following:

a. Inception report: This report will include the evaluation plan, methodology, data collection tools, and analysis plan. It should be submitted to WCC for review and approval within the first week of the commencement of the contract.

b. Draft evaluation report: This report will include the findings, conclusions, and recommendations of the evaluation. It should be submitted to WCC for review and feedback after 4 weeks (20 working days) of the commencement of the contract.

c. Final evaluation report: This report will incorporate feedback from WCC and stakeholders and should include a summary of the methodology, findings, conclusions, and recommendations. It should be submitted to WCC within 2 weeks (10 working days) of receiving feedback on the draft report. The final report should be no longer than 40 pages, including an executive summary. Summaries of data collected should be included in the appendixes, taking into consideration confidentiality and anonymity.

 

All the deliverables should be written in English.

 

 

Given the context in which the programme operates, the consultancy is highly confidential. The contractor should keep responses to survey anonymous and all exchanges with different stakeholders confidential. Materials developed will be owned by WCC and circulation will be limited. The contractor should abide to these strict confidentiality requirements and the WCC code of Conduct.

 

 

WCC staff will:

• Discuss with the selected contractor/consultant a reasonable timeline for the evaluation.

• Provide a detailed ToR.

• Provide a detailed list of all stakeholders.

• Provide the WCC-EAPPI Strategic plan 2017-2021.

• Provide the programme’s related data and contacts.

• Participate in meetings and workshops of the assessment as needed.

• Facilitate meetings and workshops with key informants and focus groups as needed.

• Manage the contract/assessment process and serve as a liaison with stakeholders.

• Have overall responsibility and accountability for the assessment.

• Approve all deliverables.

 

The contractor is responsible for:

• Conducting the assessment as per the agreed plan.

• Personnel management (if applicable), in particular identifying the need for short-term assignments within the available budget, as well as planning and steering assignments and consultants.

• The production of deliverables in accordance with contractual requirements.

• Regular progress reporting to WCC in accordance with deadlines.

 

 

 

Contractors or institutions wishing to apply must send the following:

a. Technical proposal; along with

b. Financial proposal with quote of the daily fee cost (in CHF, EUR or USD); and

c. CVs of all evaluators involved.

 

The assessment and selection of consultant proposals will follow the WCC Procurement Policy.

Only selected contractors/consultants will be contacted by the Programme Executive in charge of the EAPPI, based in Geneva, Switzerland.

 

 

This consultancy assignment implementation period is approximately 30 working days over 3 months. The assignment is envisaged to start in September 2024 and a final report must be submitted by the end of November 2024. The detailed timeline will be discussed with the selected contractor/consultant.

 

 

Qualifications of the contractor/consultant(s) must include but are not limited to: 

• A minimum of five years practical international experience working in design and implementation of monitoring and evaluation activities of similar programmes and organizations.

• Experience in programme impact evaluation. These include specific experience in quantitative and qualitative research methods, tools, and techniques, such as design and analysis of baseline and evaluation surveys, interviews.

• Experience in working with senior managers of church-based organizations. 

• Advanced working knowledge of MS Word, Excel, and/or other applications. 

• Experience in participatory methods including rights holders.

• Understanding of Rights-Based Approach, Do No Harm, conflict sensitivity and gender sensitivity.

• Sound knowledge of HRL and IHL and of the political and economic terrain and historical background of the countries and region concerned.

• Strong applied research and analysis skills, including experience of working in challenging conditions and working to agreed standards of consent and security.

• Experience of conducting and moderating focus group discussions and proven awareness of the sensitivity required in such contexts.

• Fluency in written and spoken English and Arabic is required.

• Previous experience with ecumenical/church-based programmes is preferred.

 

The contractor/consultant(s) should have the capacity to assign a qualified and experienced team that will work on this assignment, and to ensure a gender balance in the team.

 

 

The proposal must be sent by 1 PM (CET) on 26 August 2024

Please apply on the WCC career page:

Call for proposal : evaluation and impact assessment of the WCC-EAPPI (Ecumenical Accompaniment Programme in Palestine and Israel)