Impact Assessment

Impact Assessment – AMENCA and SIDA – Small Ruminant Value Chains – Terms of Reference

Background

Oxfam’s Economic (EJ) program in the occupied Palestinian territory (oPt) works towards engendering systemic agricultural market system development using the market system development approach to strengthen the economic resilience of Palestinians and enhance their capacity to adapt and react to shocks, stresses and uncertainty.

To that end, Oxfam’s oPt EJ program Theory of Change (ToC) is as follows: ‘for Palestinian women, men and youth to exercise their rights to sustainable and equitable economic development, it is first and foremost necessary to protect and enhance their resilience of livelihoods while working toward systemic change that addresses the structural barriers to economic development in the oPt.’

The impact capturing assessment will focus on the following two EJ projects:

Developing Equitable Agricultural Production and Market Systems for Resilient Economic Development in the occupied Palestinian territory, funded by the Swedish International Development Cooperation Agency (Sweden) and focusing on six value chains: cucumber (West Bank and Gaza), potato (Gaza), olive and olive oil (Gaza), grape (West Bank) and small ruminant (West Bank), and traditional food processing (West Bank and Gaza). The project’s specific objective is to Increase the resilience and income of small-scale women and men producers through improved production and market systems in the West Bank, including East Jerusalem, and the Gaza Strip.

Improving Small-Scale Producer’s Access to and Power in Olive, High Value Fruit, Small Ruminant Value Chains, funded by the Australia Middle East NGO Cooperation Agreement Program Phase 3 (AMENCA 3), through the Australian government. The project aims to connect Palestinian Farmers to markets and make agricultural value chains work better for SSPs in the West Bank and Gaza by improving market readiness, enhancing private sector engagement and trade facilitation while also promoting youth and women’s inclusion and focusing on three value chains: small ruminant (West Bank), olive, and high value fruits.

Rationale

This is an opportune time to conduct an impact capture exercise as the AMENCA project is nearing completion (scheduled project end date is September 2019), and plans are under way to develop the next phase of SIDA in 2020.  In addition to the potential to capitalize on lessons learned, the objective of assessing impact is to identify more expansive intended and unintended outcomes that go beyond quantitative indicators and capture more profound impact at a human level. That is, the aim is to articulate the more abstract level of change and impact that is evidenced by changes in attitude and behaviour.  

This call came about in due part from findings of a funder (the Australian government) led external annual evaluation of AMENCA. The evaluators identified some anecdotal evidence of outcomes during their data-collection and meetings with project beneficiaries that was not captured in the conventional annual reporting format. These findings, to some extent, substantiate long-held assumptions amongst program management that broader level impact is not being adequately captured through standard evaluation and reporting mechanisms. Based on further internal discussions on how to best address this gap in identifying and documenting impact, Oxfam decided that an external impact capture assessment would be the best course of action.  

Previous evaluations of both projects produced information at an outcome level; however, by their form, these project level evaluations are not conducive (both in scope and methodology) to capturing broader levels of impact. This is not to say that these evaluations are not important or useful, they serve a specific and necessary purpose for internal and external reporting, but they are limited to measure at the level of set project output indicators and tend to be more descriptive in nature rather than analytical.  

Given the large scale of these projects, it is more prudent to pilot this exercise through one of the value chains implemented under both projects. The small-ruminant value chain is a fitting choice as it is a key component in both projects and as such can serve as a useful case study for analysis of the scope of impact of this specific model of value-chain and the market-system development approach at large.

For example, in the case of small-ruminant value chains, we have evidence that herders increased their income, and that there was a decrease in livestock mortality rates and an increase in births by approximately 50%. Beyond some anecdotal evidence, the impact on the individual, community or sector domains has yet to be well substantiated and articulated.

As such, the mandate of this call is to go beyond the parameters of outcome evaluations and carry out an impact capture exercise that assesses whether intended and unintended long-term impact (change) occurred that can be attributed to the project, but also it should also identify why and how this impact came about.

Purpose

As stated above, the objective is to identify the level and scope of impact that came about due to the interventions’ contributions. Furthermore, the broader purpose is to ‘zoom in’ and document impact to extrapolate learning to inform program quality improvement and accountability, future program development (replicate, scale-up, end) and knowledge sharing.  To ensure that the findings are useful for this end, the materials and products of the assessment must be high level, digestible pieces with clearly delineated, practical and actionable learning and recommendation components.

The primary audience of this assessment is internal Oxfam program staff and Oxfam’s donors. However, if the exercise yields higher level findings and learning, then they will be further developed and shared with Oxfam’s global knowledge hubs.  

Key Impact Questions

The Overarching question:

To what extent does the market-system development approach strengthen the economic resilience and improves the lives of the ‘people we work with’?

The above should be unpacked using but not limited to the following questions:

• What market system changes in the small-ruminant sector in the targeted region can be attributed to the interventions?
• What change (positive or negative) in the knowledge, attitude and behaviour of the main stakeholders engaged at different levels of the market system chain took place due to the intervention?
• What were the factors that contributed to/or hindered the change?
• If change occurred how sustainable and scalable is it?
• To what extent was the project implementation effective? What were the contributing factors?
• Were all stakeholders fully engaged at all phases of the intervention?
• What is the socioeconomic return on investment?
• Was the geographic ‘cluster’ approach appropriate for the interventions and context?

Scope and Methodological Approach

The developed methodology of the impact assessment must adhere to the gender and culturally sensitive principles with a participatory approach that is inclusive of Oxfam’s staff, partners and the ‘people we work with’.

The proposed methodology must clearly articulate a clear understanding of the main question; provide a description of the phases of the approach including sources for primary data collection and the type of data analysis to be employed.

A foundational principle of a sound methodology is the use of a mixed-method approach.  While this is an expected criterion, the proposed methodology must demonstrate how qualitative tools will be utilized and analyzed to give more context and depth to the findings.

After the selection process and before signing the contract, the methodological section must be updated with a detailed timeframe.

Qualifications and skills needed

• At least seven years of experience in social development/international development and economic development utilizing market system development approach.
• A sound understanding of the social, economic and political context of communities in Palestine, including the impact of the Israeli occupation, the fragmentation of Palestinian territories and relevant policy issues.
• Proven experience in conducting impact assessments, reporting, grant management and guidelines, including experience of quantitative and qualitative approaches, participatory methods, the DAC evaluation principles and DCED standards.
• Experience in project/program management with good analytical ability in project monitoring, appraisal, and reporting, particularly working with local partners.
• Advanced and demonstrated data collection skills both in qualitative and quantitative data.
• Excellent written and verbal skills for the development of reports, tables, and presentations.
• Demonstrated ability to describe results of analysis to a wide variety of stakeholders.
• Excellent inter-personal and communication skills, oral and written.
• Fluency in verbal and written English; strong spoken Arabic language skills is preferred.
• High level of computer skills.
• A high degree of personal organization, self-sufficiency and reliability, be a self-starter, able to act with initiative but within agreed parameters, able to prioritize time and attend to detail while coping with peaks and troughs of workload.
• Ability to write clear and useful reports. A sample report of previous similar evaluation experience should be submitted with the application.
• Ability to manage the available time and resources and to work to tight deadlines.

Financial Proposal

Please submit the proposal (technical & financial offer) and requested documents by email ([email protected] ) before 06 06 2019 COB, addressed to: Nabila El-Ahmed quoting the reference code in the subject line of your email.

***Attachments: SR impact assessment Final ToR 5.21.19