Measuring outcomes across multiple youth serving organizations and programs
This interesting report gives examples how to evaluate outcomes across multiple organizations and programs. For example, one of the cases that they describe is the Strive Initiative in Cincinnati which includes Early Childhood, Mentoring, Tutoring, Afterschool, College Prep, basically a whole group of programs that have partnered together to form a continuum of youth services and to report on a core set of outcomes. It is interesting to see how mentoring fits into their overarching scheme!
The report on ways to evaluate across programs:
Breakthroughs in Shared Measurement and Social Impact, by Mark Kramer, Marcie Parkhurst, and Lalitha Vaidyanathan, FSG Social Impact Advisors, 2009
http://www.fsg-impact.org/ideas/pdf/Breakthroughs_in_Shared_Measurement.pdf
Information on the Strive Initiative:
http://www.strivetogether.com/documents/roadmap_bibliography.pdf
http://www.strivetogether.com/
Quoting from the report: “Adaptive Learning Systems also help to align organizational strategies and goals among the dozens or hundreds of organizations that influence an issue, thereby building the collective capacity of the entire nonprofit system. This is essential to solving complex social problems. Our failing public education system, for example, cannot be fixed only by improving early childhood school readiness, nor by targeting afterschool programs in middle school, nor by boosting college preparation efforts in high school. Any lasting solution must address the entire educational continuum.
Within and across this continuum, the ways in which different organizations coordinate and support each other’s work profoundly influence the effectiveness of the system as a whole. If early childhood programs aren’t aligned with kindergarten requirements, tutoring programs have no access to classroom materials or student test scores, and college preparation programs are not linked to local universities, the effectiveness of each program suffers. Even at a single point on the continuum, the lack of consistency across dozens of tutoring programs in a given city undercuts their success as students move among them from year to year.” (p. 16).

These reports describe a really important potential that is made possible by the Internet. People from within an organization, or across many organizations, can document actions which over time are intended to archive a shared goal. Breakthroughs in Shared Measurement and Social Impact provides great rationale for why systems like this should be embraced, as well as some realities of how they need to be supported for many years in order to have the benefits they promise.
I've been working with an on-line documentation system since 2000. You can view it at http://www.vattsystems.com/ohats/Home.aspx (this works best in Internet explorer).
The T/MC system was modeled after a system being used by a group at the University of Kansas. Our mission is to help tutor/mentor programs grow in all poverty areas, so this system is not specifically set up to document what happens within individual programs.
If you look at the mission statement, the first point is "collect information" about what programs operate in the Chicago region, where they are, what they do". Thus, you'll see in the documentation, and on the web site, that we maintain a database of more than 200 organizations in the Chicago region who provide some form of volunteer-based tutoring/mentoring.
The second step in the strategy is to "increase public awareness, or, get more people to look at the information. The third is help people understand the information, so (fourth step), they become a volunteer, leader, donor, etc. helping one or more programs get the resources they need to operate and grow.
Every action that we, or anyone else, takes towards these goals can be documented. More than 1300 actions have been documented since 2000.
Our system was created with a small grant of $15,000 in 2000. The funder did not provide additional money after that, so we did not have funds to keep improving the system in 2001-6, nor did we have funds to hire an evaluator to "make sense" of the data. The original system did not have automatic reporting features, so as more people documented actions the list grew longer, and less people would look at it.
Then we began to get spammed. Thus, in 2003-6 you'll see fewer actions being documented by fewer people (just me).
In 2007 a volunteer from Baltimore rebuilt the system, having work done by his company in India. What you see now is interactive. Each time someone documents and action, the metrics charts update. We can sort the list of actions by date, goal, recorder, and thus see the impact different people are having at different times. If you want to take a look you can log in using username GUEST and password VISITOR
I'm telling you this because we want other non profits, and for profits to share the strategies and goals of the T/MC, and add yourselves as recorders, to document what you do to achieve these goals.
We've set up discussion forms with each of the metrics charts,so if you want to know more about what each chart means, you can read it in the forum, and you can join in and ask questions.
This is far from a million dollar system, yet if used by many people, it might result in millions more being available to all of the different volunteer-based tutoring and/or mentoring programs in the country.
Daniel F. Bassill
President
Cabrini Connections
Tutor/Mentor Connection
800 W. Huron, Chicago, IL 60642
312-492-9614