Philanthropy and Social Justice: Thinking Differently About Evaluation

Steven E. Mayer, Ph.D. / Effective Communities LLC

Occasional Note / June 12, 2008

Our website JustPhilanthropy.org presents many productive avenues for pursuing social justice using the resources of philanthropy. Funders, nonprofits, and potential donors exploring these options frequently ask, “How can we evaluate these efforts?”

A useful big-picture metric for philanthropic efforts: disparities reduction

Social justice work has as its goal, almost by definition, to reduce disparities of the kind regularly found in regional and national data. Public records everywhere show that our society’s systems and markets perform unreasonably better for some racial/ethnic, economic or circumstantial groups than for others. Philanthropy can help change this.

Good public data exist on disparities in all walks of life (in criminal justice, education, economic opportunity, among many; see our Inventory of Disparities). Once a philanthropic organization (a foundation, nonprofit, or donor) decides to address one of these disparities, with the goal of closing the gap, it should keep its eyes on pertinent big-picture data and learn from their rise and fall. While the goal for philanthropy is to “bend the trend lines” or “move the needle,” it’s the work leading to that goal that warrants evaluation.

Philanthropy and evaluation

Funders frequently ask of nonprofit efforts, “What’s the bottom line?” Unfortunately, this question has limited use outside of business settings, where it originates. In business settings, improvements in the bottom line (profit) stem from improvements made above the bottom line in the various departments that make up the entire venture. Social justice work is similar in one essential way: improvements in the bottom line (disparity reduction) stem from repairs made to the upstream factors that contribute to that measured disparity. But in the nonprofit and philanthropic sector, unlike the private sector, this bottom line is not under the control of a single organization.

Reducing even one measured disparity (such as infant mortality, access to economic development capital, or sentencing to prison) is a task for multiple efforts in multiple settings and jurisdictions. The efforts of different entities – nonprofits, community groups, public agencies, donors, even families and individuals – need to flow together to produce change in the big-picture metrics. The point for evaluation: if no one organization alone can improve the social bottom line, certainly no one grant can do it alone. Overcoming the isolation of typically separate efforts is one of the biggest challenges facing philanthropy interested in reducing disparities. Individual grants should be evaluated (both pre-grant and post-grant) for their likely potential contributionto an improved big-picture metric, not for producing it single-handedly.

Needed: useful evaluation questions

“What is being achieved through this effort?” and “What kind of results are you getting?” are worthwhile questions that must be addressed to be fair to those who support this work. But demands for “measurable impact” and “outcome measures” are inappropriately placed on separate, local efforts; they apply more to the bigger picture, the picture indicated by disparities data. This is not to avoid the questions, but instead to find better ways of answering them. More satisfying data that inform next steps, stimulate innovation, engage participating stakeholders, and make better use of scarce philanthropic capital would come from asking for “evidence of progress” or even “early signs of impact.”

Think courtroom, not science

To appreciate these better questions, try this mental exercise: assume the program you support or operate has been accused of being trivial or ineffective, doing nothing to reduce disparities or improve social justice. What evidence could you provide in its defense? Think of a parade of witnesses testifying from their unique expertise, vantage point, experience, and vested interest. What “portfolio of evidence” could make a case good enough to persuade a jury of peers that this work, when considered in context, is useful and necessary for closing a key disparity?

No less rigorous or accountable

Asking for “evidence of progress” is by no means a diminished demand for rigor. Instead, it frames evaluation in more familiar and approachable terms. Data of all kinds can be considered -- numbers, stories, graphs, pictures, records, opinions, artifacts, etc. There is no single “measure” that communicates effectiveness or truth, just as in a courtroom no single witness provides all the testimony. In a court, multiple lines of evidence are entered and judged on their merits, resulting in conclusions that stand tests of credibility and accountability.

A structure for investment and evaluation

The Pathways to Progress outlined on JustPhilanthropy.org are designed to serve as “performance dimensions” or “criteria for evaluation.” They are guidelines of where to invest philanthropic resources, and at the same time, where to look for evidence of progress. The six Pathways show the areas that, if strengthened by philanthropic investment, stand to help close a gap or fix a disparity. In our framework, philanthropy produces more social justice when it focuses on disparities reduction and invests in these six pathways:

• Creating stronger organizations, with operations aligned with fixing disparities;
• Supporting the necessary discussions to move progress along;
• Moving creative ideas towards successful implementation;
• Strengthening relationships, networks, and leadership that promotes progress;
• Raising and deploying additional financial and human capital useful to this effort;
• Knitting these different efforts together in ways that pressure a key disparity.

The associated evaluation questions are, What evidence suggests this kind of progress is happening? What plausibly links this progress to changes in measured disparities? What plausibly links your philanthropy to this progress? Good brainstorming should allow you to compile a portfolio of existing records, pointed stories, expert opinion, traffic counts, paper trails, photographs, commissioned studies -- a scrapbook full of evidence (we like to hope) that reveals the value of this work as a contribution to reduced disparities. And looking further ahead, what does the evidence tell you (and others) of where these efforts could stand improvement?

More can be said about any of the points made above. Please tell me which points most need amplification if this communication is to be useful to you. Contact me at StevenEMayer@msn.com. Thanks!