Promoting Inclusive Development for People with Disabilities...
I. Terms of Reference of the Mid-term External Evaluation
1. Context and Purpose of the Evaluation:
Since 2011, APEFE has been supporting BASR focusing on promoting the rights of persons with disabilities in the most disadvantaged areas of the West Bank / Palestine. The 2017-2021 strategy focuses mainly on improving access to essential services including medical, rehabilitative, educational, vocational and psychosocial services. To achieve that, the focus has been given to building the capacities of BASR teams including CBR, community centers and nursing staff to ensure quality and sustainability and supported its endeavor to transform into a national resource center for disability. Started in 2011, the program has offered support for community centers to promote access to education and vocational training for children and adults with disabilities. It helped with developing a multidisciplinary team (outreach team) to reach the most disadvantaged persons with disabilities and meet their essential rehabilitative services
The program adopted the CBR strategy, which seeks to protect and promote the rights of persons with disabilities as an integral and indispensable part of the human rights system, develop service provision and remove all the barriers that hinder their inclusion in their local communities on equal footing with others. (CBR brochure in English is attached)
After multiple years of support, the program has advocated for enhancing community ownership and accountability of all relevant stakeholders in the targeted local communities. The community centers were supported to assume financial and technical responsibility through utilizing income-generating projects, mobilizing local resources, and enhancing disability mainstreaming. Therefore, the mid-term evaluation will serve as a tool to understand the extent to which local communities and BASR partner organizations can autonomously sustain their efforts and ensure the quality of life of PwDs.
In accordance with the commitments undertaken with the APEFE's local partners and the Belgian state, the interventions implemented by the APEFE are subject to mid-term and final evaluations.
The mid-term external evaluation that is the subject of this contract:
is in line with the APEFE's obligation of accountability towards taxpayers and the funding body, the Directorate-General for Development Cooperation (Belgian Federal Development Cooperation);
is in line with a desire and a responsibility on the part of the APEFE to provide reporting to its partners, target groups and beneficiaries;
is intended to contribute to internal learning and to any strategic, methodological, administrative and financial decisions and approaches to improve the results to be achieved and the sustainability of the benefits of the program.
By analyzing the outcomes of the interventions and its operating procedures, the evaluation will help identify most suitable interventions, draw lessons and good practices and develop new strategies and offer the needed support for the management and implementation team.
The evaluation aligns with APEFE and BASR commitment towards promoting the rights of PwDs. The evaluation will help program stakeholders to:
make necessary adjustments based on the recommendations,
open perspective for potential partnerships with other stakeholders,
maximize opportunities within the current context.
manage potential risks,
By drawing on the lessons learned and good practices, the evaluation will be beneficial to APEFE to improve its program formulation, implementation, and monitoring-evaluation practices.
The evaluation will be of great benefit for the CBR in terms of improving the implementation of the 5 years program, ensure quality and sustainability of the program components, and open spaces for strategic thinking and planning for future plans and interventions.
2. The Scope of the Evaluation:
The evaluation will focus on the program “Promoting Inclusive Development for People with Disabilities in the West Bank (Palestine)” and its four outputs. The evaluation will focus on the implementation period between January 2017 to October 2019 taking into account that the program has started in January 2017 and will continue until the end of December 2021.
It will focus on the capacity development provided to the direct partner, BASR, and the other stakeholders involved in the program including community centers, DPOs, SHGs, and local authorities in the southern districts of the West Bank, namely Bethlehem and Hebron.
The evaluation team will work with the following:
BASR general director
CBR program director
Community centers supervisor
Multidisciplinary rehabilitation team (Outreach Team)
Disability rights officer
CBR field supervisors and community workers
BASR staff including livelihood program and other departments.
Mental health unit staff
Persons with disabilities; and their families.
DPOs and SHGs
Community centers (one old and one new center)
Local authorities
Government directorates (health, education, social development.etc)
Local universities
UNRWA disability program
Others
3. Context and Theory of Change of the Programme:
Overall, there is a lack of commitment towards promoting the rights of PwDs on a macro and micro political level. Despite the more positive shift in attitudes towards PwDs in recent years, negative attitudes are still prevailing, particularly in the targeted communities. Given the current political situation in Palestine, imposed by the Israeli authorities, the free and uninterrupted movement is not guaranteed. PwDs face numerous challenges including mobility restrictions, which in turn, hinders their access to basic medical, rehabilitative, educational and socioeconomic empowerment services.
Political, attitudinal, institutional and environmental barriers result in the systematic exclusion of PwDs and hinder their social participation. Consequently, the majority of PwDs are unable to equally access services and rights in Palestinian society.
Theory of Change:
To induce inclusive socio-economic development, PwDs and their families should actively participate in all aspects of life. To achieve this; two basic conditions should be met:
Accessibility and use of appropriate services to provide them with decent living conditions.
Supportive settings at social, political, legal, economic, educational, informational, and environmental levels.
The change should occur at three levels:
With regard to the rights holders (disabled and vulnerable people and their families), the aim is to support them claim, defend, and enjoy their rights. This pathway of change is focused on the empowerment of PwDs and local community organizations to raise awareness on disability, to advocate for the rights of PwDs and to lobby decision makers (as duty bearers) to mainstream disability within their policies, programs, and day-to-day practices.
With regard to those responsible for upholding those rights (the government institutions), the aim is to enforce these rights where they are directly concerned and set up restrictive norms to ensure that these rights are respected by everyone involved, both individuals or groups. The program believes strongly that political changes at policy level constitute the most effective mechanism to bring about (positive) structural, long-term changes for PwDs.
With regard to society (communities, organized civil society), the aim is to provide an environment where rights holders can enjoy their rights. Civil society organizations have to remove all barriers and hold social responsibility by providing social, cultural, and economic opportunities to PwDs and their families.
Two mechanisms can ensure the fulfillment of these conditions: awareness raising and capacity building:
PwDs need to become more aware of their rights and to understand their issues in a rights-based approach. This awareness raising is a crosscutting issue that should be carried out continuously and reach out to all community members, with a focus on influential key people in the communities (staff of primary health care centers, community centers, local authorities, etc.).
Therefore, an important element of successful change will be the capacity and awareness of:
Representative groups of disabled people (DPOs) to identify, analyze and lobby for the removal of barriers to participation. From previous experience, the programme has learnt that awareness sessions in the past years have born their fruits, which is clearly demonstrated by the increased number in disability advocates, community centers, SHG members, local authorities and volunteers in the communities who are actively involved in the enhancement of the quality of life of the PwDs.
Community-based organizations (community centers, young and women centers, and local authorities, etc.). They have to assume their social and financial responsibility to provide essential services towards PwDs and to advocate for their rights to inclusive development. These local organizations are formulating the social fabrics of their areas and represent an effective advantage to promote the living conditions of PwDs and their families.
The main local partner, Bethlehem Arab Society for Rehabilitation was founded in 1960. It is a non-profit non-government organization that is nationally recognized for the comprehensive medical and rehabilitation services it renders to beneficiaries from different parts of Palestine, particularly those with disabilities regardless of their gender, age, religion or social class.
BASR has worked progressively upon its commitment to enhancing the overall quality of life of persons with disabilities and other vulnerable groups, inspired by its mission for their total inclusion into all aspects of community life. Since its inception, BASR has thoroughly developed innovative programs and services that meet the emerging needs of the Palestinian community.
BASR also contributes to the empowerment of people with disabilities, their families and DPOs to advocate their rights to full participation in society through awareness raising on disability issues, capacity building activities, joint integrated activities with DPOs as well as advocacy initiatives.
BASR renders its comprehensive services to people with disabilities based on a human rights approach, also reaching out to those living in rural underprivileged areas through its outreach rehabilitation program and Community Based Rehabilitation (CBR) program, covering the southern part of the Palestinian territories (Bethlehem and Hebron districts).
The Objective of the Programme:
"People with disabilities in the disadvantaged areas of the southern region of the West Bank (Palestine) have improved access to essential services (medical, rehabilitation, education, vocational training, and psychosocial services). The program aims at empowering persons with disabilities and their families to contribute to the inclusive development in disadvantaged areas of southern Palestine. To this end, it targets the quantitative and qualitative improvement of health, education and social components so that users of these services can benefit from better accessibility and quality services.
The Strategy of the Programme:
A team of qualified professionals (multidisciplinary team) will provide community-based rehabilitation services as well as various equipment or adaptations needed to improve the quality of life of people with disabilities and promote their wellbeing.
The program supports the development of community centers by mobilizing local communities to take their social and financial responsibility to provide educational and/or vocational rehabilitation services in the respective communities.
People with disabilities and their families, as well as the main local organizations active in the rights of persons with disabilities, will be supported to raise awareness on disability issues, defend their rights, advocate for enforcing disability-related laws and policies. Local authorities will also be supported to enhance disability mainstreaming within their policies and action plans.
The program focuses on the capacity building of the implementation teams to ensure sustainability and the wider impact on targeted groups. The organizational and managerial development will help the organization act as a national center for disability issues in the targeted area.
2. Objectives of the Evaluation:
In the context of the major changes in the Belgian and international development cooperation environment, marked by the recognition of the central role of local actors in development processes, APEFE wishes to initiate an evaluation that is based on participatory approach through:
Involve all the stakeholders of the program. They will be involved in the preparation of the evaluation (development of the ToR for the evaluation mission), the implementation of the evaluation and the communication of the results.
Assess the quality of the Theory of Change and monitoring and evaluation processes.
The evaluation is expected to take the “Theory of Change’’ (TOC) approach to determine causal links between the intervention and observed progress in the inclusion of PwDs. It is important to know whether the design of the program (TOC) is still appropriate for achieving the desired changes/benefits. In the long term, in terms of inclusion of PwDs, it will identify adaptations needed to improve the implementation of the program and achieve its desired outcomes. An assessment of the hypotheses and risks will be made to ensure that they are logical and sound and that they have contributed to determining the planned activities and products.
The evaluation has to confirm whether the system for measuring the performance of the program is compliant with TOC.
Consider the OECD's DAC criteria of effectiveness, efficiency, and sustainability.
Evaluation questions related to these criteria are listed in paragraph 5 “evaluation questions”. The evaluation will assess whether the program is on the right track to achieve its results and mitigate weaknesses/risks identified during the program formulation. The program effectiveness and its approach to capacity development are very important, in order to achieve results with maximum quality.
Draw observations and conclusions, and make recommendations.
3. Evaluation Questions:
The evaluators are expected to produce a validated evaluation report pursuant to effectiveness, efficiency, and sustainability criteria.
However, the evaluators may, subject to the approval of the support committee, propose questions or even criteria in addition to those selected in this ToR. These additions will, in this case, be added in the service contract of the evaluators.
Output 1: A comprehensive set of essential, home-based services has been ensured to PwDs in the targeted disadvantaged areas by a multi-disciplinary “Outreach Services Team”.
Effectiveness :
Does the psychosocial monitoring system adequately measure?
The quality of the services they provide.
The community’s ownership and accountability towards disability issues?
How could they improve their monitoring system to better focus on these two issues?
Output 2: A network of community centers (CC) has facilitated access to appropriate & improved educational & vocational rehabilitation services for PwDs in the targeted areas.
Effectiveness :
The main purpose is to empower and build the capacities of the centers’ staff and to review the CBRs role in providing technical support and professional supervision with regards to the following questions:
How can the CC be transformed into an inclusive educational institution that works both on providing education and promote the right to education?
To what extent do community centers act as a network? What are the benefits and missed opportunities? While considering a network dynamic allowing the centers to take better advantage of the context, how can they :
improve the links with general technical and vocational training and socio-occupational integration programs (BASR and others)?
be part of the general education and vocational training systems at the national education level (Ministry of Education)?
Sustainability :
What are the ways to contribute to the social and financial sustainability of Community Centers? How to ensure it by the end of the program?
Output 3: PwDs have been empowered to become self-advocates about their own rights to bring about positive changes in their local communities and to lobby organizations and authorities to mainstream disability policies and laws in their plans, practices, and budgets.
Effectiveness :
The main purpose is to improve the capacity building offered by the program with regards to the following questions:
To what extent the DPOs’ and SHGs’ roles, mandates and efficiency in advocating for the rights of PWDs as representatives of the disability movement need to be revisited and in which way?
To what extent the program has supported the SHGs and DPOs in empowering and supporting PWDs (and other right holders) to advocate for their rights and lobby duty bearers to promote enforcement of disability-related laws and policies? How can they improve their methods to better act according to the right based approach?
To what extent the program has contributed to empower PWDs and increase their influence on local stakeholders/authority?
How to improve the local authorities’ accountability and ownership towards disability-related issues through improving disability-mainstreaming efforts?
How to support the sustainability of the disability movement through enhancing networking among disability advocates and creating lobbying networks on government and civil society organizations?
Output 4: The organizational and human capacities of BASR have been strengthened for delivery of quality services to target groups & beneficiaries and for an effective transformation to a national resource center for the relevant stakeholders
Effectiveness :
To what extent BASR is on track to become a national resource center in the area of inclusion for PwDs, with regards to the matters of digitalization of the information as well? What are the main achievements? What are the needs and steps forwards?
How could BASR/CBR transform into a research hub for scholars, academics and other civil society organizations?
4. Methodological Approach:
The evaluation team has the responsibility to propose an evaluation methodology. The consultancy should adopt an evaluation methodology coherent with the participatory approach of the project. The consultant/s is expected to conduct a participatory evaluation providing significant involvement by the project partner, its beneficiaries and other interested parties.
The list of persons targeted by the evaluation given in chapter 2 “Scope of the Evaluation” is indicative but not exhaustive. The evaluation team will be able to put forward proposals for better functioning. They can propose a list of persons at the intervention level (local personnel, beneficiaries, authorities, etc.) they want to meet.
The consultancy should be carried out based on a desk study (Chapter 7) and field visits.
The expert will propose methods and systems of data collection and analysis.
For data collection, various methods and tools will be used:
Interviews with key partners and stakeholders
Field visits to project sites and partner institutions
Survey questionnaires where appropriate
Participatory observation, focus groups, and rapid appraisal techniques when needed.
The evaluation team will detail how the evaluation will be conducted by focusing on the subject, the objectives, and the evaluation questions.
The team will supply a draft evaluation matrix showing for each evaluation question the judgment criteria, data sources, and justification of their choices and for each source, the methods and tools for data collection and their limitations. They will justify the suitability of the proposed methods and tools.
A description of the role of the stakeholders in the evaluation against the data that needs to be collected will be provided.
The briefing will be to discuss the evaluation matrix, adapt it and provide additional information on the data to be collected. There will also be written a feedback from the evaluation support committee.
Before completion of the final version of the evaluation report, a workshop bringing together the stakeholders must take place. The objective is to give the stakeholders involved the opportunity to hear the evaluation team's findings and conclusions and to make written and oral observations.
5. Data, Information, and Documents Available:
The program started its activities in January 2017 and will continue until December 2021. The Programme Technical and Financial Document or detailed formulation document for the program and the multi-year program submitted to the Directorate-General for Development Cooperation (DGD) comprising the ad hoc Theory of Change (TOC), which will be the subject of the evaluation, is attached electronically to the Terms of Reference. In the context of the preparatory work that will follow the awarding of the contract to the selected evaluators, the documentation may be annexed with other relevant narrative and/or financial sources.
In addition, the evaluator will have access to the following information:
Agreement documents.
Minutes of the steering committee and other events relevant to the project implementation
APEFE result oriented report
Annual BASR/CBR reports.
Annual work plans
Budgetary and accounting data is available within APEFE's monitoring and evaluation system. A daily expenses report is available.
The data related to the indicators for 2017 and 2018 is available in the annual 2017 and 2018 execution reports.
CBR has an internal database with internal data from activities conducted by the CBR program.
6. Deliverables:
The following deliverables must be submitted:
An inception report containing adjustments concerning the conduct of the mission, its schedule, the updated evaluation matrix containing the evaluation questions, their judgment criteria, data sources, the method of data collection and analysis and its limitations, the target groups to meet, etc. This report will be submitted 5 working days before the start of the mission.
A PowerPoint presentation of the results of the evaluation. The consultants should consider comments and observations.
A provisional evaluation report, which will be produced within the 15 working days, which follow the end of the field mission.
A final evaluation report with a maximum of 30 pages, not including appendices, including observations by APEFE and other stakeholders, must be available within 10 working days following receipt of the comments. If these observations express assessment differences not shared by the consultants, these could be appended to the final report and commented by the consultants.
7. Stages and Milestones/Schedule:
The service (preparation – field mission – provisional report – final report) must take place :
The evaluation will be conducted beginning from mid-September until the end of October 2019. During this period, community centers will be open and time is suitable for conducting field visits and meetings with relevant stakeholders.
The total duration of the billed service, all activities included, must not exceed 32 working days.
The following delivery dates should be set in the agreement between APEFE and the evaluators, based on the timetable proposed by the latter:
Briefing with/by the APEFE, at the headquarters – (video conference is allowed according to the location of the expert);
Preparation of the logistical component of the mission (visa, booking, purchase of transport tickets, etc.);
Briefing with the BASR direction in Beit Jala.
Conduct of field visits, interviews, workshops, and focus groups.
Oral presentation of the results of the mission with the stakeholders at the end of the field mission at BASR.
Complete applications should be submitted no later than July 18, 2019, to the Procurement Department via email at [email protected].
