Final Project Evaluation - “Improving Access to the Labour M...
Date: 03/2022
TERMS OF REFERENCE FOR THE FINAL EXTERNAL EVALUATION
PARTNER ORGANIZATIONS
Johanniter International Assistance (JUH) – is a German NGO with headquarters in Berlin, Germany and working in 18 countries in Asia, Africa, Middle East and Latin America. JUH has been working in Middle East since 2009, and registered the regional office in Amman (2016). It’s has a programmatic focus on Health, Protection, education support in humanitarian context, Maternal and Child care, MHPSS Care services, WASH, Livelihood, inclusive programming with specific focus on PWDs and emergency responses to humanitarian situation in context of displacement and refugee crisis. Johanniter International Assistance has been active in Palestine in the field of health care since 2004.
Bethlehem Arab Society for Rehabilitation
BASR was first established in 1960 in the absence of a national body guaranteeing the fulfilment of access of persons with disability to the right to rehabilitation. As a non-governmental non-profit organization, BASR seeks to achieve an inclusive society for all. BASR adopts the rights-based and community-based inclusive development approach. Today, BASR works in three major fields, which are medical surgeries, specialized rehabilitation, and community-based rehabilitation.
The Community-Based Rehabilitation (CBR) Program aims at enabling society to abandon the marginalization and exclusion of PWDs. It marks a historical transformation point for tackling disability issues, since it represents a comprehensive model that works on different aspects to ensure that PWDs enjoy all their rights. This could only be made possible through PWDs’ active participation in the change process, leading to inclusive development for all.
BACKGROUND AND RATIONAL
Despite the fact that there has been a dedicated legislation governing the rights of PWDs in Palestine since 1999 (The Palestinian Disability Law No 4 for the year 1999), as well as dedicated articles under different Palestinian laws, the everyday reality of PWDs in various aspects remains challenging. PWDs’ access to their rights is rather limited as these relevant laws were unable to achieve the intended transformation. This is mainly due to the fact that these laws are not fully effective, and they lack reinforcement and sanctions on those who violate them.
According to PCBS statistics, disability prevalence is estimated at around 7% in Palestine. However, the actual percentage is expected to be much higher, because PCBS has used the limited definition of the medical model of disability. Furthermore, some families tend to hide information about members with disabilities, especially females. As a result, many PWDs go without being registered.
The most common type of disability in Palestine is physical disability, followed by learning difficulties. Statistics also show that over 53% of PWDs in Palestine are illiterate, 87.3% are unemployed, 34.1% are single, 75% are unable to use public transportation, and around 15% of PWDs under 15 years old dropped out of school. Furthermore, there is a structural limitation in PWDs’ access to needed rehabilitation services, medical consumables, assistive devices, and medical equipment. Additionally, there is a lack of early detection systems.
The CBR approach has been adopted in Palestine since the 1990s through the CBR Program at Bethlehem Arab Society for Rehabilitation. Since its beginnings, the program has targeted PWDs from all types of disabilities, age groups, and gender, as well as their families, Disabled People’s Organizations (DPOs), governmental and civil society organizations in the West Bank areas.
BASR adopted the CBR approach since 1993 and established the CBR program to facilitate the access of people with disabilities to their basic rights of rehabilitation, economic independence, education, and so on. Throughout the years, BASR and its CBR Program have successfully and continuously worked with people with disabilities, their families, and their local communities and local councils to include them in all aspects of society, including the economic aspect.
PURPOSE, OBJECTIVES AND USE
The main objective of this evaluation is to provide BASR, Johanniter and BMZ/bengo with an assessment of the project, its design, implementation, and results. The aim is to determine the relevance and fulfilment of objectives, efficiency, effectiveness, and impact of the project. The evaluation should provide information that is evidence-based, credible and useful, enabling the incorporation of lessons learned into the future decision-making processes and projects.
The evaluation will specifically:
1- Assess the extent to which the project met planned results.
2- Assess the extent to which partnerships were formed during implementation of the project.
3- Highlight lessons learnt, best practices and recommendations for improvements to feed back into current and future BASR programming in the same sectoral areas and using similar approaches to meeting their objectives
4- Assess satisfaction of beneficiaries with project’s interventions.
PROJECT DESCRIPTION:
Starting in December 2018, the project aimed at improving the socio-economic conditions of people with disabilities and to increase their access to employment. Therefore, the project also addressed service providers, companies, institutions, political decision-makers, and the public in order to enhance awareness on the rights of PWDs and create inclusive environments. The project was implemented in partnership with Johanniter International Assistance and funded by Federal Ministry for Economic Cooperation and Development (BMZ), bengo-Engagement Global.
The project focuses on the development of a sustainable intervention that address the economic rights of people with disabilities in a more holistic approach. It applies a twin-track approach. On the individual level, the project facilitates the access to career counseling and vocational assessment, which is followed by capacity building interventions, such as vocational training courses, soft and employability skills training, and micro-enterprise management training. Additionally, the project ensures access to assistive tools and equipment to enhance access to employment. On a broader level, throughout the project, roundtable meetings were conducted with representatives of various institutions to promote the rights of people with disabilities to decent work, with a focus on the social development fund component.
The number of direct target group is 500, consisting of people with disabilities, private sector, as well as representatives of relevant ministries. A total of 250 men and 250 women with disabilities are targeted from all West Bank areas, to acquire the necessary qualifications to be competitive in the job market. This is through soft skills training, technical, and professional capacities. Another 50 representatives from the Ministry of Social Development and the Ministry of Labor are targeted to participate in professional training. The relevant ministries are also targeted in regard to lobbying to promote the SDF and its implementation mechanisms.
At the same time, one of the components of the project is the delivery of various assistive devices to enhance the fulfilment of rehabilitation needs of people with disabilities, which impacts their access to vocational training and employment. To further enhance the vocational assessment component, the project equipped CBR/ BASR with Valper Unit and allowed BASR to establish is own vocational assessment unit.
– Further Description
Project Activity Number Achievable points
Establishment of the Employment Unit and recruitment of a full-time employment officer N/A To achieve systematic job matching between unemployed PWDs and potential employers looking for qualified applicant.
Vocational career counseling for people with disabilities 500 PWDs To help create vocational identity
Vocational training courses for people with disabilities 150 PWDs To acquire PWDs with the needed vocational skills for potential employment
Life skills training for people with disabilities 200 PWDs To acquire PWDs with employability skills, also known as soft skills, such as CV writing, job interviews, job search, basic report writing, team and management skills.
Management training – formation of small companies 60 PWDs To acquire PWDs with micro-management skills.
Carrying out of round table meetings 50 representatives from private sector To raise awareness on the value of PWDs’ employment
Establishment of an advisory committee consisting of representatives from MoSD and MoL 15 meetings To participate in the SDF’s development and implementation.
Performance of a national study in relation to the SDF (social development fund) 7 workshops of 25 persons To support the development and implementation of the SDF.
Lobbying work 3 TV Programs
3 video adverts
3 infographic videos
4 advertised social media posts
1 advertising board
3,000 text message To increase awareness
Launching the Social Development Fund 1 To start the implementation of the SDF
SDF training for ministries 2 trainings for 50 persons To share knowledge on regulations and mechanisms of promoting the employment of PWDs.
Kindly see the attached file in which the logical framework is explained.
SCOPE OF EVALUATION
The evaluation will focus on the impact of the project “Improving Access to the Labour Market for People with Disabilities and Strengthening their Economic Participation in the West Bank” during the implementation period of December 2018 to April 2022 all over the West Bank. The evaluation should assess the extent to which partnerships have been established, capacities have been built, and that implemented activities have achieved the results intended for the project.
It will also assess the implementation strategy itself and recommend areas for improvement and learning towards achievement of objectives aligned with Johanniter’s and BASR’s missions.
Methodological notes
BASR suggests consideration of the following mixed-method methodology in order to collect the relevant data. The consultant is expected to determine the final methodological approach for presentation and approval during the inception phase. Final approval will be made BASR’s focal point.
The evaluation is expected to be based on the findings and factual statements identified from review of relevant documents including the project documents, monthly, quarterly and interim reports, case studies, KAP survey, and so on. BASR will provide the external expert with all available project documentation at the beginning of the consultancy.
Project specific context shall also be taken into account. The consultant will also undertake field visits and interview the stakeholders, including the target beneficiaries. Participation of stakeholders in the evaluation should be maintained at all times, reflecting opinions, expectations and vision about the contribution of the project towards the achievement of its objectives. Gender-balance should be taken in consideration for all persons/groups participating in field visits.
The methodology must consider participants’ safety throughout the evaluation as well as research ethics (confidentiality of those participating in the evaluation, data protection, gender, age and ability-appropriate assent processes) and quality assurance (tools piloting, enumerators training, data cleaning). The above-described methodology is indicative. The consultant is expected to provide a detailed methodology and work plan. He/she will also be free to collect additional data in order to reply to all the research questions.
The consultant must conduct field missions to obtain the necessary qualitative and quantitative data that provides evidence of the impact of the response with members of communities targeted by the project. The evaluation should be conducted mainly through secondary data review, focus group discussions, key informant interviews, case studies, etc.
Throughout the evaluation process, the evaluation team must consider important issues such as gender, environment, accountability, and other ethics like ‘do no harm’.
Deliverables
The evaluators are expected to submit the following deliverables:
– Provisional and final evaluation reports
– All data and data analysis results submitted throughout the evaluation: focus group results, meeting minutes, survey data… etc.
DATA COLLECTION
The collection of field data will take place in the different governorates in which the project took place. Evaluators must use all necessary means to collect the data they need. These include interviews, focus groups, questionnaires, observations, documents and reports, and other necessary tools.
The final evaluation report must be written in English and must include comprehensive qualitative and quantitative analysis based on the various datum collected.
EVALUATION STAGES AND TIMEFRAME
The service must take place starting from 5 April to the end of April 2022.
The following delivery dates should be set by mutual agreement between Johanniter, BASR, and the evaluators, based on the timetable proposed by the latter:
Deliverables/ Evaluation Stage Actors Involved Month
Briefing meeting with/by the Johanniter-BASR BASR Johanniter April 2022
Desk review analysis of all relevant data including but not limited to the documents sent by partners to the evaluation team. Desk review analysis must also consist of required desk research to be done to provide evaluators with the needed context for further analysis. BASR April 2022
Conduct field visits, interviews, workshops, focus groups, and so on. Partners will provide the contact information needed for coordination in this regard. BASR
Stakeholders and beneficiaries April 2022
Submission of the provisional report based on the provided templates (Attached) April 2022
Submission of the comments on the provisional report April 2022
Submission of the final report April 2022
The expected number of working days is 27. The Evaluation report should be structured as indicated in the translated Annex 7.2. from bengo/BMZ called “Outline for Evaluation reports” (see in Annexes/Attachments below)
EVALUATION QUESTIONS
The evaluators are expected to produce a substantiated evaluation report for the DAC criteria: Effectiveness, efficiency, coherence, relevance, sustainability, and impact. The questions under each are only guiding principles.
However, the evaluators may, subject to the approval of the support committee, propose questions or even criteria in addition to those selected in this ToR. These additions will in this case, be added in the service contract of the evaluators.
Efficiency
1. To what extent did the project ensure that resources were used in operational strategies that prove to bring about desired changes?
Coherence
2. To what extent were the project interventions and implementation strategies complementary and coherent to bring about desired changes? What procedures and measures did the project consider to avoid overlap or duplication?
3. To what extent did the project interventions align with national strategies and priorities?
Relevance
4. To what extent did the project best respond to developments/changes due to the COVID-19 crisis?
5. To what extent were the project and its interventions relevant to address the needs of people with disabilities as identified in the targeted communities with special attention to women?
Effectiveness
6. To what extent are the results achieved in capacity building for people with disabilities sufficient to enhance access to employment and livelihood opportunities?
7. How effective has the project been in responding to the needs of the beneficiaries, including emerging needs during the implementation period?
8. How satisfied were the targeted PWDs (M, F) and various institutions about the project’s interventions (in terms of access to training and employment, and promoting the right to decent work)?
Sustainability
9. How are the skills gained from training going to impact the possibility of PWDs’ future employment?
10. Could successful models for replication be identified and promoted?
Impact
11. What changes did the project bring into lives of targeted men and women with disabilities and their families during its implementation? Can we make links between the changes observed and the contribution of the project?
Requirements Profile for the Evaluation Team
The evaluation team is expected to consist of at least two experts. The evaluation team should be independent.
Profile of the team leader:
– An academic degree preferably in social sciences or any other relevant field.
– At least 10 years of professional experience.
– At least 3 similar evaluations of institutional or civil society projects in relation with human rights and/or disabilities in the MENA region. In particular on conducting evaluations of projects, preferably with knowledge and previous experience in project evaluations for bengo/BMZ.
– Knowledge of community-based development strategies
– Knowledge of the rights-based approach to development
– Knowledge of the needs/challenges of working with vulnerable populations, specific knowledge of disability and development will be considered a plus
– Have sound knowledge of evaluation and data-collection methods, familiar with the participatory approach;
– Knowledge of result-based management;
– Mastery of basic office software (Word, Excel, PowerPoint);
– Must be able to work independently with excellent with excellent demonstrated team work, coordination and facilitation skills.
– Be able to communicate effectively in Arabic and English.
Profile of the second consultant:
– An academic degree in social sciences or any other relevant field;
– At least 3 years of professional experience and previous experiences in disability issues in Palestine
– Good experience in project evaluation and familiarity with the participatory approach in the MENA region. In particular on conducting evaluations of projects, preferably with knowledge and previous experience in project evaluations for bengo/BMZ.
– Knowledge of the rights-based approach to development and knowledge of the needs/challenges of working with vulnerable populations, specific knowledge of the disability and development will be considered a plus
– Be able to communicate effectively in Arabic and English.
MANAGEMENT OF THE EVALUATION
The role of BASR:
BASR will involve the principal stakeholders to support the evaluation process. During preparation for the mission, they will:
i. Inform all local stakeholders of the activities planned by the evaluators (survey, consultation, meeting, etc.) to ensure their availability;
ii. Supply the evaluators with the required documents for desk review;
iii. Supply the evaluators with the needed documents and information to perform their mission;
iv. Organize a briefing meeting with the evaluators at the start of their mission in the West Bank;
v. Organize a debriefing meeting on the conduct and qualitative content of the evaluation at the end of the mission in Palestine;
vi. Review and share feedback on draft reports;
vii. Distribute the provisional report for feedback;
The role of the evaluation team:
i. Present complete and unbiased information, so that the decisions or measures taken are well founded;
ii. Protect the anonymity and confidentiality of the individuals who communicate information to them; complying with data protection and safeguarding policies provided by BASR and APEFE.
iii. Pay attention to:
– Beliefs, usage and customs and demonstrate integrity and honesty in its relationships with all the stakeholders.
– Any form of discrimination and gender disparity present in the evaluation, its limitations, findings, and recommendations clearly, precisely and credibly and encourage goodwill towards the involved actors.
– Present the evaluation, its limitations, findings and recommendations clearly, precisely and honestly and encourage goodwill towards the involved actors
iv. They will be responsible for the performance and quality of the evaluation. They will ensure:
– Reliability of the data by an ad hoc triangulation;
– The multiplicity of program internal and external sources of information;
– That the methodology makes clear the links between data, findings, analyses, and conclusions;
– That the recommendations are realistic and operational by specifying, for example, the target groups for the recommendations, their timeframe and the actions to undertake to implement them.
QUALITY OF EVALUATION
The evaluation service and the final report will be subject to a quality evaluation by Johanniter and BASR.
CRITERIA FOR BID SELECTION
Bid selection will be carried out based on the following criteria:
– Criteria 1: Methodology; understanding and interpretation of the terms of reference and the context; relevance and realism; compliance with the timeframe
– Criteria 2: Financial bid
– Criteria 3: Profile of the evaluators (training, skills, experience) and modalities of work
– Criteria 4: Clarity and presentation of the bid
The content of the bid:
I. A technical offer including:
– An understanding and interpretation of the context and terms of reference
– The description of the process, the approaches, and the methodology to perform the evaluation
– The composition of the team, if any: presentation of the evaluators, their experience, and modalities of work.
– A proposed detailed timeframe for the final evaluation.
II. A financial offer detailing the budget (eligible costs: fees including all charges, intellectual output, local subsistence expenses, expressed in people/day, transport expenses, taxes, and any other required evaluation costs) in New Israeli Shekels;
III. An annex including up-to-date curriculum vitae including a list of previous similar evaluations, and the administrative documents requested.
Consultants are invited to submit their technical and financial offers no later than Wednesday 13th of April 2022 to [email protected]. Offers should be signed and stamped, and submitted on official letterhead document where applicable.
Any request for additional technical information should be addressed to [email protected].
Annex/Attachments:
– Outline of the evaluation report
– Logical framework for the project
More at
https://basr.org/eng/2022/03/29/terms-of-reference-final-external-evaluation/
