Give a man a fish, he eats for a day. But if you lobby for better fishing policy…

What’s the most cost-effective way to help people?

Lots of people have tried to answer that question. Donors care, a lot, about whether the charities they’re giving to are effective (though they often don’t know a good way to measure that). Charity evaluators like GuideStar, CharityWatch, and Charity Navigator rate charities primarily by how well they’re run.

My favorite charity evaluator, GiveWell, takes a different approach — the group focuses on identifying charities that deliver the most cost-effective interventions available. In global health and development, for example, we often have robust evidence that some health interventions are a particularly good way to save and change lives. That means that the charities delivering those interventions are places where your money goes especially far. Those interventions include distributing malaria nets, treating kids for intestinal parasites, supplementing Vitamin A to reduce child mortality from infectious disease, and, yes, just directly giving people money.

But can we do better than that? That’s a question GiveWell is asking — and it is expanding its scope in the hopes of finding even more promising opportunities.

GiveWell recently announced that it’s more than doubling the size of its research team to try to find more cost-effective programs. But its revised approach involves increased attention to something relatively new for GiveWell: policy-oriented philanthropy.

In a blog post announcing the change, the organization said it would be researching new, more complex ways to measure how to do good in areas, including:

GiveWell is not planning to change their focus on low- and middle-income countries. But they’ll be considering lots of interventions they wouldn’t have looked at before.

This is a big deal for people like me who follow GiveWell closely. But it’s of broader interest, too. Handing out cash, or distributing health care where it’s needed, is in some sense the kind of aid we actually know how to do right. Working toward better institutions and better policies has potential to do far more good, but it’s also a lot more complicated — and past work has often gone badly.

Here’s one way to think about what GiveWell’s up to.

A charity that distributes malaria nets can do good with each net it distributes. But a charity that persuades a local government to change a law that’s affecting malaria net distribution might be able to do as much good, with less money. Obviously, a complete solution to global poverty will involve a lot of both approaches. But up until now, GiveWell has largely — though not entirely — focused on the first approach.

Some recent exceptions have been noteworthy. In 2017, GiveWell recommended a grant to the Center for Pesticide Suicide Prevention, which identifies the pesticides most commonly used in suicides and advocates that governments ban them. In 2018, it recommended a grant to the Innovation in Government Initiative, part of the Abdul Latif Jameel Poverty Action Lab (J-PAL) at MIT, which provides technical assistance to governments trying to scale up proven antipoverty programs, as well as partnering researchers with national governments interested in answering questions around national policy.

To most people who are used to thinking about American policy, it’ll be obvious that charities can sometimes do more good by lobbying the government than by trying to directly address the problem with their own resources. Homeless shelters do important work, but ending homelessness is going to involve changes to local, state, and federal policy to help the poor — nonprofits don’t have anywhere near the budget to do it all themselves. Feeding the hungry is important, but you might get more leverage from helping them sign up for food stamps, or convincing their state to expand eligibility for food stamps so that they qualify.

But here’s the thing — it’s a lot easier to measure the effects of a charity that works directly on a problem than the effects of a charity that works indirectly, on lobbying. The homeless shelter can tell you how many people it housed. The food kitchen can tell you how many it fed. But an advocacy organization will typically have a harder time describing the exact role it had in bringing about new legislation.

In addition to that, there’s good reason to be concerned about lobbying and activism as a form of charity in poor countries. Aid programs in the developing world have saved millions of lives, but there’s also been plenty to criticize — including a trend of rich Westerners assuming they have more knowledge and expertise than the locals, and of programs being run by Westerners being assumed to have transformative potential that local activist groups and local institutions are presumed not to have.

Of course, there are ways to avoid those pitfalls. Advocacy groups that have learned from the history of coercive or badly designed aid programs tend to emphasize that their role is providing locals with the resources they need to make policy decisions themselves. Here’s how the Center for Pesticide Suicide Prevention describes their process, for example:

We work with local and regional governments to first find out what is known about HHPs [highly hazardous pesticides] in their country and then train local researchers to identify the suicide, accidental poisoning, and environmental consequences of HHP use. We provide modest funding for, and assist, these groups to collect the necessary data and provide it to both national regulators and international organisations. We work with pesticide regulators and other decision-makers to help them formulate recommendations and implement policy reform aimed at phasing out HHPs.

I don’t think it’s impossible to do lobbying and advocacy work internationally in a responsible way that empowers local researchers and enables governments to make the decisions that best serve their own populace. But there’s no doubt that it’s significantly more complex to identify the best ways to do good in this space than it is to identify the best ways to distribute bednets.

I went down to GiveWell’s offices to talk with them about the expanded focus. One thing I wondered was why it hadn’t happened sooner.

“When GiveWell started,” Catherine Hollander, research analyst at GiveWell, told me, “Elie [Hassenfeld] and Holden [Karnofsky] were just working in finance and didn’t have specific expertise in assessing charities or philanthropy. And so that really led them to focus on the measurable, RCT [randomized-controlled-trial]-style evidence that is so associated with GiveWell — because that’s where they really felt that they could assess and understand the impact of charities and to also make that case publicly.”

Seen from that perspective, the work GiveWell has been doing for the last 12 years — in-depth as it is, and it can be astoundingly in-depth — was all aimed at answering a much simpler question than the one I started this article with: “What’s the most cost-effective way to do good?”

That question is simply too ambitious for a new organization just getting off the ground, and requires too much specialized expertise, research background, and experience for a small team to get right. So GiveWell specialized instead in answering the question: “Which charities are delivering the interventions with a robust evidence base suggesting they’re a highly cost-effective way to do good?”

That’s an important question. There was a clearly a niche for it — GiveWell’s research is relied upon by thousands of donors in their annual charitable giving, and GiveWell has moved more than $100 million to the charities they recommend.

But GiveWell has now answered that question pretty thoroughly, with lots of top charities that are all “in a fairly narrow band of cost-effectiveness,” Hollander told me. “Many of our top charities we estimate to be something like five to 10 times as cost-effective as cash transfers.”

GiveWell’s standard research focus and process were very good at finding charities in that range of cost-effectiveness. It started to seem likely that they weren’t going to find something significantly better with the same approach. “Our best guess for where we find something that’s much much more cost-effective is in the space of something like policy advocacy,” Hollander said.

Thus, the expansion of GiveWell’s focus.

The new, expanded mission will involve making grant decisions that involve more “subjective judgments” than GiveWell’s past research, the blog post observes. That’s because those judgments are hard to avoid when you’re trying to estimate the effectiveness of advocacy or technical assistance. If GiveWell doesn’t have the background and rigor to make these subjective judgments well, then their new grants might not be nearly as cost-effective as their current top charities (which they plan to keep listing in the near term).

But in truth, there’s always been a lot of subjectivity in cost-effectiveness estimates — GiveWell’s, as well as any other estimates. Is saving the life of a baby as valuable as saving the life of an adult? More valuable? How should we compare pain and suffering to loss of life? What about loss of income?

When you’ve looked at these questions enough, it starts to feel unprincipled to say that certain kinds of subjective judgments are welcome in our cost-effectiveness estimates (in fact, impossible to avoid), but other kinds of subjective judgments — say, how much of the credit for this law passing is it reasonable to assign to this charity? — aren’t.

Yes, the new research will involve more subjective judgment. But comparing any two programs has always involved a whole lot of subjective judgment — maybe more of it than we’re comfortable admitting. Once an organization gets into the comparison business at all — and once they have the expertise to work in more complicated areas without making destructive mistakes — they might as well focus their judgments on the big questions.

Sign up for the Future Perfect newsletter. Twice a week, you’ll get a roundup of ideas and solutions for tackling our biggest challenges: improving public health, decreasing human and animal suffering, easing catastrophic risks, and — to put it simply — getting better at doing good.