8th Annual Monitoring and Evaluation Report 2020
Many Rivers is on a 10-year journey to understand and strengthen the economic and social value created for individuals, their families and wider communities through annual evaluation reports. Each year Deloitte Access Economics reports the outcomes and impacts achieved by Many Rivers by drawing on quantitative and qualitative data.
“Many Rivers is supporting clients and communities to move out of structural disadvantage via a journey of growing self-belief, accountability, networks and business acumen.” Deloitte Access Economics
Read a summary of the 8th Annual Monitoring and Evaluation Report here
In 2012, Many Rivers engaged Deloitte Access Economics to develop a framework to measure the long-term performance of Many Rivers’ Microenterprise Development program.
The framework study looked at:
- the types of outcomes likely to be associated with Many Rivers’ activity;
- the data and indicators needed to assess the outcomes; and
- recommendations for how the data and indicators should be collected.
In developing the framework, the likely economic and social outcomes associated with Many Rivers’ program were considered at a number of levels:
- at the individual level, for those receiving support (both financial and non-financial) from Many Rivers;
- at the business level, for those receiving support (both financial and non-financial) from Many Rivers;
- at the community level, for those in which Many Rivers operates; and
- the flow-on implications for the Australian economy more broadly.
The framework developed by Deloitte set out the details required to measure the short-and long-term outcomes associated with Many Rivers’ program. A range of potential indicators were included and relate predominantly to economic, social and capacity building outcomes for individuals, businesses and their communities.
Details on these outcomes and suggested indicators for their measurement are supplemented by practical information that will guide Many Rivers to develop its approach to data collection.
The original framework has been continuously developed each year:
- In 2013, Many Rivers implemented a comprehensive data tool – ‘Compass’ – which embeds data collection for evaluation in their customer relationship management system, enabling a data driven and increasingly innovative understanding of Many Rivers’ outcomes.
- From 2014, Many Rivers has collected Client Stories, interviewing clients about their journey while working with Many Rivers. These stories provide an additional level of impact fidelity, and understanding of the conditions for success in microenterprise development.
- In 2018, the evaluation included the reporting of broader social outcomes being achieved by Many Rivers’ clients, and the reporting of the Many Rivers Community Economic Development Program.
- In 2020, Deloitte Access Economics has undertaken the 8th annual evaluation to assess the activities and impacts to date. As a result of COVID-19 and the associated restrictions, Many Rivers has seen changes to its new and existing client base. The evaluation expands on the progress of the previous years by:
- Providing more insights on the client journey by analysing the types of client interactions with Microenterprise Development Mangers (MEDMs).
- Incorporating insights about the Community Economic Development (CED) Program from interviews with CED Managers (CEDMs).
- Understanding the results in the context of the broader impact of COVID-19 on the Australian economy. There is also shift to reporting longitudinal statistics, as opposed to point-in-time measures.
Previous Annual Evaluation Reports (2012-2019)
- 8th Outcomes Evaluation (2020) – Full Report
- 7th Outcomes Evaluation (2019) – Full Report
- 6th Outcomes Evaluation (2018) – Full Report
- 5th Outcomes Evaluation (2017) – Full Report
- 4th Outcomes Evaluation (2016) – Full Report
- 3rd Outcomes Evaluation (2015) – Full Report
- 2nd Outcomes Evaluation (2014) – Full Report
- Evaluation Summary Report (2013)
- Evaluation Summary Report (2012)
Role of Deloitte Access Economics
Deloitte Access Economics has worked with Many Rivers on the Annual Outcomes Evaluation since 2012, commencing with the creation of an Evaluation Framework and followed by seven annual outcomes evaluations, drawing on information of increasing quality and breadth in each subsequent year.
The purpose of monitoring and evaluation is to assist Many Rivers with understanding the nature of the aggregate and year-on-year impact of its programs.
In this context, Deloitte Access Economics does not fulfil the role of an auditor. Assessment of the quality and completeness of the data that Many Rivers collects is beyond the scope of this work. Rather, Deloitte Access Economics provides an updated analysis of, and information on, the progress of the Microenterprise Economic Development program. It includes an in-depth analysis of the characteristics of clients that Many Rivers supports, analysis of the nature of their businesses, as well as an analysis of socio-economic outcomes produced as a result. It also includes the reporting of information on the Community Economic Development program.
Although over the years, the focus has been on monitoring the performance of Many Rivers’ programs, the evaluation of their impact is becoming an increasing feature of this work.