The Impact Assessment of SMS Programs and Services in Farming Communities

Understanding how our programs and services measure up

Metrics are tantamount to a compass in business. They show us whether we are moving forward or backward. They orient us and guide us toward new pathways. To that end, ECOM has adopted a new methodology to assess the impact of SMS’ (ECOM Sustainable Management Services) programs and services.

Much of SMS’ success is due to time and effort spent with farmers in their communities, allowing our field staff to tailor the best solutions. However, as farmers’ needs evolve over time, so should the way we measure success and well-being. Such robust data will allow SMS to revisit its solutions and tailor them to greater effect, if needed so that we can understand and capitalize on social progress — and failure — to the betterment of farmers and their communities.

Impact Assessment: An exhaustive approach

ECOM is in the process of developing an adapted Impact Assessment methodology to monitor the social, environmental and economic impact of its services on farms and producers. In partnership with The Latin American Center for Competitiveness and Sustainable Development (CLACDS), a Costa Rica-based applied research center. The methodology and tool under development will gauge the effectiveness of SMS’ programs and services, and business management practices.

Through its collaboration, SMS is also aiming at integrating new, innovative measurement systems such as the Social Progress Index in the methodology. By adapting measurement approaches such as the SPI, ECOM will be able to better monitor and evaluate the impact and changes over social aspects of producers and their households. Through the digitalization of the tool, each SMS local team around the world will be able to apply the surveys and monitor impact and progress.

“The Impact Assessment tool focuses on the different services local SMS teams are giving and to the conditions they can control,” said Benjamin Rimaud, Environmental & Social Officer.

A data gold mine

The Impact Assessment’s primary objectives are to measure the improve in the quality of life of farmers and producers while increasing the accountability and sustainability of the supply chain. The interpretation of the data will be contingent upon understanding environmental, social and economic factors, including drought, political tensions and market price. ECOM’s data-driven online platform, SMS Integrity, will house all of the information, providing analyses, like social return on investment within an integrated dashboard.

New technologies in data collection and analysis will enable SMS to further expand its project opportunities by providing greater visibility into the impact clients are having through shared partnerships and projects, while its measurement of key indicators can be used for marketing purposes. Above, all, this data should expand SMS’ capacity to create partnerships that best meet the needs of the farmers, producers and their surrounding communities.

Works in progress

Trying to measure the impact of SMS is complex, given the different supply chains and local contexts. Our key objective is to retain the quality of the data through a standardized questionnaire, clear communication regarding assessment process and execution, and comprehensive training of local data collectors. Nevertheless, it is vital that the impact assessment approach be somewhat flexible.

"The tool should be able to focus or adapt to a certain degree to represent [each] context in the best way," said Benjamin Rimaud. "Otherwise, you have a blanket assessment that is not representative. The key challenge is to understand where that limit is [and] at which point it changes the whole nature of the assessment."

The SMS pilot in Peru in 2019-20 underscored these challenges. To complement first learning, the impact assessment will be tested in additional supply chains and eventually scaled up to all countries of origin. Ultimately, these metrics aim to provide data that moves beyond profit and speaks to social progress. Each trial brings us one step closer to an inclusive tool that will fine-tune our interventions so that farming communities can be more productive and resilient.