A flexible approach to impact assessment

In the social investment space, measuring project outcomes and impact is often considered only at the end of a project lifespan to highlight the work that has been done. However, this loses the true value of impact measurement, which is the early identification of bottlenecks and developmental areas, and thus the opportunity to course-correct based on data. Our experience as Tshikululu has shown us the value of planning for impact measurement and data collection right from the beginning of a project to ensure that future assessments are built on a strong foundation. In reality, however, this is not always possible. We have found that we need to be flexible in how we conduct impact assessments, acknowledging any limitations while we seek to meaningfully measure impact.

Lessons from a Global Scholarship Programme

Between 2023 and 2024, Tshikululu conducted an impact assessment for a global scholarship programme that provides financial support, mentorship and skills development opportunities to high-performing law students from developing countries. During the assessment we relied on the best practices we have developed over the course of 25 years.

Our Quadrant of Assessment

Our understanding of best practice when it comes to impact measurement is embedded in our quadrant of assessment:

Plan for measurement: An ideal impact assessment starts before project implementation even begins. It starts with a well-designed and clearly articulated project strategy and objectives which inform which areas of the project require monitoring and measurement.
Define the baseline: Effective impact measurement relies on accurate and relevant baseline data. This is used to conduct comparative analysis against data collected later on to assess impact.
Standardise data collection tools and methods: Standardising what data will be collected and how allows for the most comprehensive comparative impact assessment.
Allow for engagement: Factoring in direct engagement with stakeholders enhances the usefulness and quality of data collected, adding context and clarity to the findings.

Overcoming Data Limitations

As we have previously explored in relation to fit-for-purpose M&E, best practice is useful only insofar as it aligns with reality. This particular project presented an unusual challenge for Tshikululu: rather than designing and conducting a data collection process, we were tasked with working with pre-existing data. Scant baseline data was available, and the information gathered across stakeholder groups was not always comparable. We were nevertheless able to provide insight and recommendations to help strengthen the programme.

Key Findings and Recommendations

Ultimately, the assessment found that the programme had contributed to improving the beneficiaries’ confidence, which has helped them access opportunities and participate more actively in the workplace. We also made recommendations about how to strengthen the mentorship component of the programme in the future, and the types of networking and support most needed by beneficiaries.

Meeting Clients Where They Are

Tshikululu’s services are tailored to our clients’ diverse and unique needs. We meet them where they are to provide the best possible solutions – this includes adapting our approaches where necessary. While we always recommend that social investors embed impact measurement practices during project development and follow through consistently, we understand that the reality is seldom straightforward. Assessing impact, to whatever degree the situation allows, provides the information needed to improve performance and maximise impact. This project is an example of the value we see in navigating potential limitations to produce the best possible outcome, rather than seeking to measure impact only in perfect conditions.