FOSEK End-line Evaluation Final Report 2021

Overview

This end-line evaluation covers FOSEK, a seven-year public-private partnership co-financed by the Netherlands Ministry of Foreign Affairs (FDOV/RVO) and private partners. Solidaridad led the programme with Nestlé, the Coffee Research Institute, CMS, SMS (ECOM), C. Dorman and the Yirgacheffe, Sidama and Oromia cooperative unions. It aimed to improve food security for 120,000 smallholder coffee farmers in Kenya (9 counties) and Ethiopia through better coffee productivity and quality, diversification into food and tree crops, and a dairy pilot in Kenya. The evaluation was required by the donor and also served learning for future PPPs. Fieldwork took place in August–September 2021, and the report is dated December 2021.

Key findings

• Through a three-tier training model the programme trained 928 ToTs/technical assistants (585 Kenya, 343 Ethiopia), 137 extension officers (Ethiopia), 3,289 promoter farmers and 118,830 farmers; the report notes figures differ by source.
• 33 nurseries were built (22 Kenya, 11 Ethiopia), selling 3,319,083 coffee seedlings and 294,782 tree-crop seedlings; an estimated 48,000 farmers (27,000 Kenya, 21,000 Ethiopia), about 40% of the 120,000 target, received seedlings, partly because nurseries started late (2018 Kenya, 2019 Ethiopia).
• Evidence on coffee production is mixed: uncleaned midterm survey data show 272% growth in coffee output in Kenya and 37% in Ethiopia between 2014 and 2019, interviewed farmers reported yields rising from about 2 kg to 10 kg per tree, yet 44 trained Kenyan cooperatives reported 2–5 kg per tree (no change from 2014) and total Coop/Union production stabilised or declined.
• Coffee quality improved from grade 3 to grade 1 in both countries according to interviews, helped by stainless-steel drying tables provided to cooperatives; no reliable data on farmer sales and income were available.
• The Kenyan dairy pilot in three cooperatives with milk coolers was the most successful component: average milk delivered per farmer rose from 3 to 10 litres/day in one year, and at one cooperative daily deliveries rose from 1,000 to 7,500 litres (September 2020 to August 2021).
• Food security did not improve significantly: only chronic food deficit (more than 4–5 months) decreased, the food-poor-month peak grew in Kenya and stayed similar in Ethiopia (2014–2019); intercropping did not increase in Kenya and decreased in Ethiopia, while shade-tree use in Ethiopia rose from 38% (2014) to 65% (2020).
• Women's participation in training was low (an estimated 20% by FOSEK staff to 30% per cooperative data), financial services were essentially absent, and M&E data were incomplete; farmer satisfaction with cooperatives ('fair' or 'good') rose from 74.5% to 80.9% in Kenya and from 75.9% to 87.8% in Ethiopia between baseline and midterm.

In figures

Recommendations

• Limit the scope and ambition of complex PPPs to what partners can deliver, aim for outcomes and impact rather than maximum output, and avoid including competing partners.
• Carry out a risk assessment at design, include it in monitoring, and redo it regularly, including expected changes in partner relevance, staff and context.
• Assess potential partner cooperatives at the start to identify likely performers and specific capacity needs, and continue to give cooperative capacity development a prominent role.
• Develop markets for diversification crops from the start, and include financial institutions as partners to support financial sustainability and scale.
• Set up an M&E system managed by the programme itself with secured core budget, and plan a mid-term evaluation half-way that measures progress and is comparable with the baseline.
• If gender transformation is a core concern, give it dedicated effort, staff and resources.
• Use simplified, picture-based training materials in local languages that farmers can access themselves, and build close ties with policy makers and other extension actors.

Lessons learned

• The assumption that better knowledge alone would motivate Kenyan promoter farmers did not hold; lack of compensation and formal recognition led to drop-outs, while in Ethiopia a simpler two-tier model with lead farmers seen as farmers was more effective.
• Perennial crops such as coffee, macadamia and avocado take 3–4 years to yield, so late seedling distribution could not show results within the programme period.
• Pilots work when learning and future integration are built in, as with the Kenyan dairy pilot, unlike the Nestlé nutrition (FFN) project, which was not integrated into the programme.
• Technologies outside partners' existing production systems (e.g. vegetable intercropping) need considerable programme input to succeed, whereas tree crops that fit traditional coffee systems were adopted more readily.
• Contributions from multinational partners can change with corporate strategy, so funding should not depend heavily on pledges of future contributions.

Methodology

The end-line evaluation was mostly qualitative, carried out in August–September 2021 through interviews with FOSEK staff, partners, cooperative managers, ToTs and key informants, focus group discussions with farmers in selected cooperatives, and document review, with country reports prepared by local consultants. Findings were cross-checked against the 2014 baseline (Avance Impact) and 2020 mid-term evaluation (DEFT) survey data and validated in a sensemaking session in late October.

Limitations

• COVID-19 prevented international consultants from travelling, so fieldwork was done by local consultants and many interviews were remote.
• The evaluation budget was cut half-way because of unforeseen taxes, which reduced the scope and caused delays.
• The programme had limited, non-systematic monitoring data and documentation, and sources often gave different results, which complicated triangulation.
• Cooperatives and FOSEK staff (in Kenya in particular) selected the farmers, with trainers present in FGDs, which may have biased results positively; partners in the Yirgacheffe region could not be visited for safety reasons.
• Midterm survey data did not use the baseline template and were not properly cleaned, so the baseline–midterm comparison was limited.