Editor’s Note, April 1, 5:00 pm ET: The interview in this piece was conducted when NASA first revealed the crew for Artemis II in 2023. With the launch now taking place, Vox is republishing the piece.

The crew taking part in the Artemis II launch includes two historic firsts: the first woman, Christina Koch, and the first person of color, Victor Glover, to go on a lunar mission. Hailed by NASA spokespeople as “pioneers” and “explorers,” they have been greeted with fanfare befitting “humanity’s crew.”

But behind the Artemis II program are much more corporate goals. It’s not just that private industry helped build the program’s spacecraft. Space mining companies competing for government contracts want to turn the moon into a cosmic gas station. The vision is to mine the lunar surface for rocket fuel that can then propel us all the way to Mars — and beyond, as humanity takes its self-appointed place in the stars.

Mary-Jane Rubenstein told me that vision makes her want to throw up. A Wesleyan professor of religion and science in society, she’s the author of the book Astrotopia: The Dangerous Religion of the Corporate Space Race.

What’s “religion” doing in that title, and why is a religion professor writing a book about the space program? Rubenstein argues that today’s corporate space race — helmed by Elon Musk, Jeff Bezos, Richard Branson, and others who propose to “save” humanity from a dying planet — is actually rehashing old Christian themes that go all the way back to the 15th century, when European Christians colonized the Americas. Remember how Donald Trump described the Artemis mission and eventual settlement of the moon and Mars? He called it “America’s manifest destiny in the stars.”

But as Rubenstein points out, not everyone thinks it’s the moon’s destiny to be strip-mined, or Mars’s destiny to be settled by human colonists. In fact, some believe these celestial bodies should have fundamental rights of their own.

I talked to Rubenstein about the fear of screwing up space like we’ve screwed up Earth: Is that really a fear of trampling on space’s own intrinsic value, or is it more a fear about human nature? A transcript of our conversation, edited for length and clarity, follows.

Sigal Samuel

When you see news about space exploration, like the announcement about who will be going to the moon next year, is your dominant feeling … excitement? Dread?

Mary-Jane Rubenstein

It’s a little bit of dread. Because I worry that all this is getting going before the public really understands what’s happening.

One thing I’m worried about is that some of the astronauts will be tokenized to make it clear that Artemis is a feminist and anti-racist movement. But if we’re looking to make space exploration a liberationist project, just putting representatives of different identity groups there isn’t going to be enough. I worry that it’ll look like the job is somehow done because there is a woman and a person of color on this mission. 

The mission itself needs to be analyzed from a feminist and anti-racist perspective first. Then you figure out how to do it well, and then you figure out who’s going to be on it. 

Sigal Samuel

There are two words you use to refer to the corporate space race in your book, and the rationale for using those words might not be obvious to readers. You talk about it as “religion” and as “colonial.” Why?

Mary-Jane Rubenstein

What I’m arguing is that the new corporate space race is an extension and intensification of the initial space race of the late ’50s and into the early ’70s. And that that space race is an extension and intensification of the colonial project that settled the Americas.

The journey that Europeans made across the seas to conquer the Americas and then the journey that white-descended Americans made across the North American continent through what’s known as Manifest Destiny gets extended in the mid-20th century as a new frontier is proclaimed to be open, the frontier of outer space. The space race is a new chapter in European-style colonialism — a vertical extension of that colonial project — as an effort to get more land and more resources for an imperial nation. 

The colonial project that settled the Americas was underwritten at every major turn by religious language, religious authorities, religious doctrines. Perhaps most profoundly, the reason Spain was able to conquer the New World was that Pope Alexander VI declared that the New World was his to give — and he gave it to Spain. The conquistadors were underwritten by the head of the Roman Catholic Church; therefore God was endorsing the Spanish conquest of the New World.

This language gets taken up in different ways later. You find a claim to land and resources and a justification for destroying indigenous communities, all authorized by biblical claims. North America is understood very early on to be what early preachers will call God’s New Israel. Just as God gave the Land of Canaan to the Israelites on the proviso that they make it a holy land, God was now giving Europeans a new Canaan. The idea is: Go in there, cleanse it of all unholiness and devotion to any other gods, and establish a new kingdom dedicated to the glory of God.

By the way, there are 20 towns in the US that are named New Canaan.

Sigal Samuel

So if America is understanding itself to be God’s new Israel, it’s like saying Americans are God’s new chosen people. How do those religious themes underwrite the modern corporate space race?

Mary-Jane Rubenstein

When Mike Pence spoke to space-industry professionals [in 2018], he quoted Psalm 139 and said that “even if we go up to the heavens, even there His hand will guide us.” Then in 2020, Trump used the language of Manifest Destiny in his last State of the Union address when he was declaring his priorities for a second term. This was in the [beginning] of the pandemic, people were dying, and his first priority was going to the moon — to embrace “America’s manifest destiny in the stars.”

That was a call-out to the old idea of Manifest Destiny, that God wants light-skinned people of European heritage to inhabit not only the Eastern seaboard but the entire continent. Now the idea that Trump set forth was, it’s not just the continent that God wants America to have, it’s the entire universe.

Sigal Samuel

And just to be clear, lest people think this is just a Trump thing, this is very much something that the Biden administration has decided to continue, right?

Mary-Jane Rubenstein

Absolutely. There’s absolutely no difference between the Trump administration and the Biden administration when it comes to space. 

Sigal Samuel

Some people object to using the word “colonialism” in this context. We think of colonialism as a hugely harmful thing mostly because European colonizers were coming to inhabited lands and destroying indigenous peoples. But if the moon or Mars or space beyond our solar system is uninhabited, how does “colonialism” apply?

Mary-Jane Rubenstein

The answer that seems compelling to you totally depends on your frame of reference. Perhaps the most difficult for secular, white Westerners to take on would be this. If you talk to indigenous people — I’m thinking particularly of Inuit cosmology, of Ojibwe cosmology, of Bawaka cosmology from Australia — they will tell you that outer space isn’t empty at all, that it actually is inhabited, that there are indigenous people there: their ancestors.

For the Bawaka People, when people die, they’re actually carried up into the Milky Way alongside the stars. So they’re really concerned that if we mine there, we’re actually doing damage to the habitation of the ancestors. And planetary bodies are often said to be sacred or to be divinities themselves. So, from different perspectives, it’s not just a foregone conclusion that there is nothing out there.

If that doesn’t do it for you, colonialism was also fairly destructive for the nations who were doing the colonizing! At the moment we do not have a robust international legal structure in space. If you’re able to set up, say, a mine there, you’re going to have to defend your mine. So the US Space Force is going to be stationed around the mine to make sure nobody else goes there. And suddenly you’ve got the same clamoring for land and resources that tore the nations apart in the late 19th century, and we had two world wars resulting from that. It seems like a bad idea to set ourselves up for that in space.

Also, the pursuit of wealth and explosion of profit tends to make those who are already wealthy much wealthier. We know that widening the gap between exceedingly rich people and exceedingly poor people is not good for most of the population. 

Sigal Samuel

What about ways that an extractive approach to space could potentially do damage to land?

Mary-Jane Rubenstein

This approach means we’re going to get even more rocket launches than we currently have — Elon Musk sends 60 satellites up at a time — and more launch pads being created and those are usually created in spaces like wetlands. Boca Chica, Texas, for example, has been absolutely destroyed by the operations of SpaceX in that area. Ecologically it’s a disaster. And low-Earth orbit is already so crowded that it’s very hard to see the stars, even for astronomers.

The next thing to point out is that the colonial project has been destructive not only of communities, but of land itself. So then the question becomes whether the land of the moon or Mars has any value in itself, which is to say beyond its value to us.

Sigal Samuel

You write about a group of Australian scholars who argue that it’s not okay to damage the surface of the moon or pollute it, that the moon “possesses fundamental rights.” They’ve even issued a Declaration of the Rights of the Moon. This echoes the “rights of nature” movement, which has successfully won legal personhood rights for lakes and forests. Do you think it makes sense to apply that sort of thinking to an extraterrestrial body?

Mary-Jane Rubenstein

It’s such a hard question. On the one hand, I can understand that people might think there are severe limitations to applying human-derived rights language to natural formations. We might be concerned that modeling the rights of nature on the rights of humans only allows us to value something insofar as it seems human-ish. But my sense is that we’re working within a complicated and insufficient legal framework and that any strategy that works is worth trying.

Sigal Samuel

You cite the philosopher Holmes Rolston III who argues that natural entities have their own value independent of anything humans might want from them. That doesn’t mean we should never eat a carrot or dig up a weed, but it does mean we should spend time considering what we take from the world and how. Rolston offers criteria for how to know when we shouldn’t destroy something. For example, we should respect “places of historical value,” “extremes in natural projects,” “places of aesthetic value,” and “places of transformative value.” 

But are these really about a place’s intrinsic moral worth? To me this sounds more like people grasping for language to talk about instrumental worth — what certain places do for us.

Mary-Jane Rubenstein

I think it’s very hard to measure the value of something in itself. We’re always going to slip into the language of human perspective; we’re always going sneak in our own aesthetic criteria. This project has really demolished anything like academic purism in me. I think we’re going to have to give up on purity, inviolable categories or absolute measurements.

But even if there were just some kind of attention to the landscape itself and to what’s important to us (taken broadly) about that landscape … even if we were just to approach the bodies of outer space in the ways that we approach national parks, where you carry out anything you bring in … we would be doing a lot better. 

Sigal Samuel

I like this idea of human judgments as a floor or a minimum. Even if we just are thinking about how to protect a place vis-à-vis what is of instrumental worth to us humans, that’s already going to be some improvement.

I think a lot of people are painfully aware of how humanity has screwed up the Earth. And so maybe there’s this fear about screwing up space. But is that really more of a fear about human nature, as opposed to really being about space’s own intrinsic value?

Mary-Jane Rubenstein

I don’t think it’s so much a panic with respect to human nature as it is a panic with respect to capitalist nature. It’s not all of humanity that wants to conquer the stars; it’s a destructive subsection of humanity that claims to be speaking on behalf of all of humanity and telling us that either all of humanity is going to become extinct forever or we need to nuke Mars [to terraform it, per Musk’s ideas]. It’s a false zero-sum game.

Sigal Samuel

Right, everyone from Musk to Bezos to Branson says the corporate space race will be for the benefit of humanity. This goes back to Eisenhower, who said the US must develop a national space program “for the benefit of all mankind.” I’ve seen this in the AI race too — OpenAI, for example, says its mission is to ensure that artificial general intelligence “benefits all of humanity.”

In your book you take issue with this language of saving all humanity from going extinct, and you write, “The operative fallacy here is known as longtermism.” Longtermism is a controversial spinoff of a social movement called effective altruism, but you say it’s actually a high-tech version of what Malcolm X called “pie in the sky and heaven in the hereafter.” He blamed America’s racist social system on the Christian teaching that those who suffer on Earth will be rewarded in the afterlife, which he said dissuaded Black Americans from overthrowing their oppressors. How does that map onto your worry about longtermism?

Mary-Jane Rubenstein

In the book I try to expose the clearly religious heritage of colonialism and the remnants of the kind of thinking we find in Pence and Trump when they say that God wants us to conquer the cosmos. But that’s not the most interesting place that religion is showing up at this point. The most interesting place is much more subtle: It’s in the proclamations that “the world is coming to an end.” They’re offering us a classic messianic logic of impending disaster on the one hand and eternal salvation on the other.

So the locus of religious operation has changed from the Church to these private messiahs. The private messiahs aren’t speaking in the name of any recognized religion — the logic claims to be totally secular. But it actually looks a lot like the Christian logic that says suffering on Earth is justified because there’s going to be redemption in another world. 

Sigal Samuel

So is your worry that the longtermist doctrine prioritizes the existence of our species in the far future, so it risks propping up the current destructive systems and keeping us docile about them?

Mary-Jane Rubenstein

Absolutely. Longtermism gives us a recommended sacrifice of the poor, homeless, and hungry of the Earth, because they’re not the future. It’s actually worse than the Christian promise. The Christian promise is that you yourself may suffer for 80 years but you will be rewarded in the afterlife. Here, there’s no reward for the particular people who are suffering. They’re just going to be thrown by the wayside and die in conditions of poverty and misery. But the human species itself will triumph.

Sigal Samuel

The human species will see the Promised Land but the individuals of today will languish in the desert. 

Mary-Jane Rubenstein

Exactly.

Sigal Samuel

Last question for you: If space exploration can be done in a way that doesn’t screw over people or animals or our planet or other planets, are you all for it?

Mary-Jane Rubenstein

I’m absolutely for it! And there are so many teachers who know how to do this better. They may not be astronomers and they’re probably not corporate leaders. But there are people who know how to live sustainably. If we can find a way to listen to their example, then great! But that would involve locating those people and probably trying this out on Earth first.

The most unique thing about human beings is this: We are creatures who long to matter.

That’s according to Rebecca Newberger Goldstein, the philosopher and author of a new book called The Mattering Instinct. If you’ve ever wondered why we humans are so singularly obsessed with discovering the meaning of life, this book — and her ideas — are for you.  

Goldstein presents an evolutionary explanation that starts off with a law of physics: the law of entropy, which basically says that things naturally tend toward disorder and destruction over time. All biological creatures need to devote a huge amount of energy and attention to resisting entropy — to surviving. But humans also have a special ability to self-reflect, and we can’t help but notice that we ultimately devote the vast majority of our attention to ourselves. To our own thriving, not the thriving of others. And so we feel the need to somehow justify that.

This, Goldstein says, is why we developed the “mattering instinct” — the drive that pushes us to find a “mattering project” that makes our lives feel purposeful and worthy. Goldstein sketches out four main ways people try to do that. 

Some are transcenders, who seek to matter to a transcendent presence like God. Others are socializers, who find purpose in helping and mattering to other people. Then there are heroic strivers, who push themselves to achieve excellence in the domain that matters to them, whether it’s intellectual, artistic, athletic, or moral. And finally, there are competitors, who focus on mattering more than others.

In the newest installment of my Your Mileage May Vary advice column, I suggested that Goldstein’s “mattering map” (see below) can be a useful tool for anyone who’s worried that AI may soon replace them in an arena where they find meaning, like their career. Locating ourselves on the map can help us each think afresh about which of the four categories makes us feel a sense of purpose, so we can consider additional types of work that could form a satisfying mattering project for us in the future.

I was curious about how Goldstein is thinking about automation-induced joblessness, what she’d do if her own work gets automated, and whether she thinks we’re in danger of losing our human dignity. So I asked her for a follow-up chat. Here’s a smattering of our nattering about mattering.

You argue that our drive to matter is one of the cornerstones of human life. What convinced you of that? How have you felt that drive show up in your own life?  

I really feel justified in my righteous anger when people treat me as if I don’t matter! 

I have a very favorite story about that. I mean, just being a woman, there are a lot of stories. But I was once at a party in Princeton with a bunch of physicists, and one very, very prominent physicist wanted to talk to another prominent physicist, and I was in the middle. So he just picked me up — I’m very slight — he picked me up and moved me like I was a potted palm!

And I had this real sense of…but I’m a person! I matter! That feels justified. And if I can justify that about myself, I have to universalize it to everybody. There’s no way it’s going to work for me and not work for everybody else.

Wow, that’s pretty appalling! 

So from that, you offer this evolutionary account of how everybody ended up with a mattering instinct. I always find it hard to evaluate evolutionary stories because there’s an element of speculation in them. Your account about how we evolved the mattering instinct seems plausible, but I could also imagine another account being true. For example, maybe the drive for mattering is a way of making sure that others will think we matter, because we want society to think well of us and take care of us. What convinces you that your account is more likely than others?

To me, it explains more of the variety of ways that people try to go about this. If the more social story were true, we would all be socializers. But I mean, the fact that there is a very strong religious aspect — I spent a good part of my life as a transcender — means that to me, phenomenologically, it doesn’t ring true. And it doesn’t ring true to the diversity [of how different people find mattering]. 

But it might be that I’ve just spent too much time with mathematicians who don’t give a damn about social acceptance!

Yes, we can see that from their fashion! But seriously, I have to say that I really love the mattering map in your book. I feel like I’m mostly one of the artistic-intellectual strivers, but I’m also a bit of a socializer in that I derive meaning from helping others with my work. Do you think most people live on only one island?

No, I don’t think so. I know that I don’t.

And I think all of us have a strong need for connectedness — it’s the other part of flourishing. We need people in our lives, and we often want to make a difference in people’s lives.

Maybe we have our main residence, and then we have our vacation home. You can definitely make a bridge [between the islands].

Why is the island of transcenders exclusively populated with different religions and spiritual traditions? I can imagine other sorts of people — like artists or psychedelic users — who feel there’s a transcendent dimension to the universe, and who derive their sense of mattering by tapping into that.

I think in some sense, all heroic strivers have some notion of the transcendent. They often talk in terms of these ideals. I mean, every artist I know talks about beauty. For knowledge workers, it’s knowledge.

But I really wanted to single out the ones who actually feel that there is some sort of personal presence in the universe that has intentions — that there’s an intentionality that permeates the universe. It’s just so very different.

I had a very religious childhood — I was brought up Orthodox [Jewish] — and it was like, God knows if I cheated and took a bite of a Hostess cupcake! And there was this sense of mattering, that I was created for a purpose. I really felt like I had a role to play in the narrative of eternity. God has his plan, and I’m part of it. And I know that when I went from believing that to not believing that, the universe changed in such a big way for me. It just felt a little meaningless, to tell you the truth. That [form of mattering through transcendence] seemed worthy of its own continent on the map.

You suggest that humans are the only animal that has a mattering instinct — we are “creatures of matter who long to matter.” You also call us “dust with dignity.” How does the mattering instinct connect with the idea of human dignity?

We are wired to take ourselves very seriously — the bulk of our attention is going to somehow be self-referential — and then we ask ourselves for justification. We feel we have to come up with some project, some story, and we devote so much energy to this justificatory project. I find that there’s a certain dignity in that. There’s something estimable, there’s something noble about a species that needs to prove to itself that it really matters.

That leads me to a very timely question: What happens to human dignity if AI replaces us in an important area, like our jobs, which is how many of us carry out our mattering projects? Are we in danger of losing our dignity, or is that some inalienable quality that we’ll just end up expressing in other ways?

The latter. I really think that when one is not able to minister to this, to appease this [mattering instinct], you end up with death within life, which is what extreme chronic depression is. So we will come up with something.

Here’s me at my most optimistic: I think about philosophy, because I’ve been speaking to a lot of philosophers who were worried about it. There’s a lot of shit work that’s done in philosophy, and yes, let AIs do it. Let them explain the 53 ways of interpreting Kant’s deontological argument. They’ll be able to do it and come up with all the utilitarian counterarguments and all of that. 

But there’s still so many problems that I think come out of being human and knowing what it’s like to be motivated by the mattering instinct and how hard it is to live an ethical life, given how much attention we are wired to pay to ourselves. AI can’t do that for us. So what I’m thinking in my most optimistic moments is that the deepest questions, they’re still going to belong to us.

I think plenty of people could listen to this conversation and say, “I don’t get my meaning from my job. What is this obsession with your career? Maybe it’s great if AI takes your job because you’ll finally learn how to find mattering in ministering to others or something!” Should we perhaps start thinking more expansively about where we find our sense of mattering?

Yeah, I think it’s not a bad idea to be thinking about that. But I also think you can’t force mattering strategies on people. It comes from something very deep — temperament, interest, passions, all of this. I’ve always resented it very much when people say, well, this here is the meaning of life.   

So I really want to be a pluralist about this. I do think that there always are going to be heroic strivers. There are people who have to meet or at least approach certain standards of excellence, including ethical and athletic and artistic.

With the artistic — just as when you have a forgery of a great painting and it’s indistinguishable from the original, it’s just not as valuable because it doesn’t come out of a human experience that came out of somebody’s individuality and what they’re struggling with — maybe that extra thing is always important in our aesthetic pleasure. If an AI writes something and it’s comparable to Shakespeare, I don’t believe that our aesthetic pleasure is going to be the same. It’s about knowing: Oh, this is a window into somebody else’s subjectivity!

In my recent advice column, I suggested that even if AI takes your job, you can hang onto a sense of mattering by looking at the mattering map, identifying the broader island of mattering that tends to make you feel satisfied, and seeing what other jobs might be an expression of that. If you yourself weren’t able to work as a philosopher and novelist anymore, what would you do instead to make ends meet while still fulfilling your drive for mattering?

There are two careers that I’ve often thought, Gee, I should have given them more thought. One is to work with children. I just love kids and I think they’re really fascinating. I have a daughter who’s a clinical psychologist, and she deals with a lot of kids, and I think it’s really interesting work. And it is that socializer [drive], which is very strong in me as well. 

The other thing is to go to Africa and just live with animals, observing [them]. I love elephants, I love chimpanzees. And I could see doing that too — a more scientific career.

This is reminding me that ever since I was a kid, thinking of humanity makes me think of an injured animal — I always pictured a three-legged dog. It’s struggling, it’s limping along. And I feel like our search for meaning is that limp. It’s a burden on us, in a way, right?

Yeah, it’s hard to be a living thing. It’s that much harder to be a human and to want to get it right. You can think of that as our limp. But you can also think of it as our crown.

For me it’s precisely because humanity is saddled with this sort of struggle that I’m rooting for it extra, that I feel a special affinity for it. 

That’s almost a protectiveness. And that’s a beautiful emotion. I mean, that is something to cultivate: Wherever there is humanity, there is a struggle, and that matters. 

American AI companies love to say that the US must win the AI arms race, or China will. 

Anthropic, OpenAI, Google, Microsoft, and Meta have all invoked the threat of a Chinese victory to justify speeding ahead on AI development, seemingly no matter what. The argument is simple: Whoever pulls ahead in building the most powerful AI could be the global superpower for a long, long time. China’s authoritarian government suppresses dissent, surveils its citizens, and answers to no one. We cannot let that model win.

And to be clear — we shouldn’t. The Chinese Communist Party’s human rights abuses are real and horrific, and AI technologies like facial recognition have made them worse. We should be scared of a scenario where that becomes the norm. 

But what if authoritarian rule that uses tech to surveil people in alarming ways is already becoming the norm in the US? If America is shape-shifting into the bogeyman it critiques, what happens to the case for racing ahead on AI?

This is the question everyone should be asking now that the Pentagon has blacklisted Anthropic — and embraced its rival, ChatGPT-maker OpenAI, which was more willing to accede to its demands. (Disclosure: Vox Media is one of several publishers that have signed partnership agreements with OpenAI. Our reporting remains editorially independent. Future Perfect is funded in part by the BEMC Foundation, whose major funder was also an early investor in Anthropic. They do not have any editorial input into our content.)

The US Department of Defense is already using AI powered by private companies for everything from logistics to intelligence analysis. That has included a $200 million contract with Anthropic, which makes the chatbot Claude. But after the US used Claude in its January raid in Venezuela, a dispute erupted between Anthropic and the Pentagon.

The two redlines Anthropic insisted on in its contract with the Defense Department — that its AI shouldn’t be used for mass domestic surveillance or fully autonomous weapons — represent such fundamental rights that they should have been uncontroversial. And yet the Pentagon threatened that it would either force Anthropic to submit to full and unfettered use of its tech, or else name Anthropic a supply chain risk, which would mean that any external company that also works with the US military would have to swear off using Anthropic’s AI for related work. 

When Anthropic didn’t back down on its requirements, Defense Secretary Pete Hegseth followed through on the latter threat — an unprecedented move, given that the designation has previously been reserved for foreign adversaries like China’s Huawei, not American companies.

As a journalist who’s spent years reporting on China’s use of AI to surveil and repress Uyghur Muslims, learning of the Pentagon’s threats reminded me of nothing so much as China’s own policy of “military-civil fusion.” That policy involves compelling private tech companies to make their innovations available to the military, whether they want to or not. Either wittingly or unwittingly, Hegseth seemed to be borrowing directly from Beijing’s playbook.

“The Pentagon’s threats against Anthropic copy the worst aspects of China’s military-civil fusion strategy,” Jeffrey Ding, who teaches political science at George Washington University and specializes in China’s AI ecosystem, told me. “China’s actions to force high-tech private companies into military obligations may lead to short-term technology transfer, but it undermines the trust necessary for long-term partnerships between the commercial and defense sectors.”

To be clear, America is not the same as China. After all, Anthropic was able to freely voice its opposition to the Pentagon’s demands, and the company says it’ll sue the US government over the blacklisting, which would be unthinkable for a Chinese firm in the same situation. But the US government’s embrace of authoritarian conduct is undeniable. 

“Racing” to build the most powerful AI was always a dangerous game; even AI experts building these systems don’t understand how they work, and the systems often don’t behave as intended. But it’s even more dangerous to try building that powerful AI under the Trump administration, which is increasingly proving itself happy to bully American companies in order to preserve the option of using AI for mass surveillance and weapons that kill people with no human oversight.

Those who are still bought in on the idea that the US must win the AI race at all costs should now be asking: What’s the point of the US winning if the government is going to create a China-like surveillance state anyway?

At least one of the major AI companies is not taking this question seriously.  

What’s really in OpenAI’s deal with the Pentagon — and why many are now boycotting ChatGPT

OpenAI announced that it had struck a deal to deploy its AI models in the Pentagon’s classified network — just hours after the Pentagon blacklisted Anthropic. 

This was extremely confusing.

Sam Altman, the CEO of OpenAI, had claimed that he shares Anthropic’s red lines: no mass surveillance of Americans and no fully autonomous weapons. Yet somehow Altman managed to cut a deal that, by his account, didn’t compromise either of them. Apparently, the Pentagon had no problem with that.

How is that possible? Why would the Pentagon agree to OpenAI’s terms if they’re really the same as Anthropic’s?

The answer is that they’re not the same. Unlike Anthropic, OpenAI acceded to a key demand of the Pentagon’s — that its AI systems can be used for “all lawful purposes.” On the face of it, that sounds innocuous: If some type of surveillance is legal, then it can’t be that bad, right?

Wrong. What many Americans don’t know is that the law just has not come close to catching up to new AI technology and what it makes possible. Currently, the law does not forbid the government from buying up your data that’s been collected by private firms. Before advanced AI, the government couldn’t do all that much with this glut of information because it was just too difficult to analyze all of it. Now, AI makes it possible to analyze data en masse — think geolocation, web browsing data, or credit card information — which could enable the government to create predictive portraits of everyone’s life. The average citizen would intuitively categorize this as “mass surveillance,” yet it technically complies with existing laws.

For Anthropic, the collection and analysis of this sort of data on Americans was a bridge too far. This was reportedly the main sticking point in its negotiations with the Pentagon.

Meanwhile, take a look at an excerpt of OpenAI’s contract with the Pentagon, and you can see in the first sentence that it is allowing the Pentagon to use its AI for “all lawful purposes”: 

You might be wondering: What are all those other clauses that appear after the first sentence? Do they mean your fundamental rights will be protected?

Altman and his colleagues certainly tried to give that impression. But many experts have pointed out that they don’t guarantee that at all. As one University of Minnesota law professor wrote: 

In fact, as several observers noted, the contract clauses call to mind what an Anthropic spokeswoman said about updated wording it had received from the Department of Defense at a late stage in their negotiations: “New language framed as compromise was paired with legalese that would allow those safeguards to be disregarded at will,” she said. 

OpenAI did get some assurances into the contract; the company’s blog post says it’ll have the ability to build in technical guardrails to try to ensure its own red lines are respected, and it’ll have “OpenAI engineers helping the government, with cleared safety and alignment researchers in the loop.” But it’s unclear how much good that’ll do, given that the impact of technical safeguards is limited and the language doesn’t guarantee a human in the loop when it comes to autonomous weapons.

“In terms of safety guardrails for ‘high-stake decisions’ or surveillance, the existing guardrails for generative AI are deeply lacking, and it has been shown how easily compromised they are, intentionally or inadvertently,” Heidy Khlaaf, the chief AI scientist at the nonprofit AI Now Institute, told me. “It’s highly doubtful that if they cannot guard their systems against benign cases, they’d be able to do so for complex military and surveillance operations.”

What’s more, “Nothing in the contractual language released up to this point seems to provide enforceable red lines beyond having a ‘lawful purpose,’” said Samir Jain, the vice president of policy at the Center for Democracy & Technology. “Embedding OpenAI engineers does not solve the problem. Even if they are able to identify and flag a concern, at most, they might alert the company, but absent a contractual prohibition, the company could not have any right to require the Pentagon to halt the activity at issue.” 

OpenAI and Anthropic did not respond to requests for comment. OpenAI later said it was amending the contract to add more protections around surveillance.

Perhaps if Altman did not already have a reputation for misleading people with vague or ambiguous language, AI watchers would be less alarmed. But he does have that reputation. When the OpenAI board tried to fire Altman in 2023, it famously said he was “not consistently candid in his communications,” which sounds like board-speak for “lying.” Others with inside knowledge of the company have likewise described duplicity. 

​​Even Leo Gao, a research scientist employed by OpenAI, posted:

For now, only a minuscule portion of OpenAI’s contract with the Pentagon has been made public, so we can’t say for certain what guarantees it does or doesn’t contain. And some aspects of this story remain murky. How much of the Pentagon’s decision to replace Anthropic with OpenAI was due to the fact that OpenAI’s leaders have donated millions of dollars to support President Donald Trump while Anthropic’s Amodei has refused to bankroll him or give the Pentagon carte blanche with the company’s AI, earning him Hegseth’s dislike and Trump’s insistence that he leads “A RADICAL LEFT, WOKE COMPANY”?

While these uncertainties linger, public mood has turned against OpenAI with nearly the speed of the tech itself. A public campaign called QuitGPT launched last month and has gained immense traction since the Pentagon clash, urging those who feel betrayed by OpenAI to boycott ChatGPT. By the group’s count, over 1.5 million people have already taken action as part of the boycott. 

It’s no coincidence that Anthropic’s chatbot, Claude, became the No. 1 most downloaded app in the App Store over the weekend, with users seeing it as a better alternative to ChatGPT.

Historian and bestselling author Rutger Bregman, who has studied the boycott movements of the past, was one of those who felt fired up upon seeing the QuitGPT campaign. He has since become its informal spokesperson. 

“What effective boycotts have in common, in my view, is that they’re narrow, they’re targeted, and they’re easy,” Bregman told me. “I looked at the ChatGPT boycott and was like: This is exactly it! This is the first opportunity to start a massive consumer boycott in the AI era, and to send an incredibly powerful signal to the whole ecosystem, saying, ‘Behave, or you could be next.’” He suggests switching over to the chatbot of any other AI company, except Elon Musk’s Grok.

Mind you, it’s worth noting that Anthropic itself is no dove. After all, the company has a deal with the AI software and data analytics company Palantir, which is infamous for powering operations of Immigration and Customs Enforcement (ICE). Anthropic is not opposed to all forms of mass surveillance, nor does it seem to be categorically opposed to using its AI to power autonomous weapons (its current refusal is based on the fact that its AI systems can’t yet be trusted to do that reliably). What’s more, it recently dropped its key promise not to release AI models above certain capability thresholds unless it can guarantee robust safety measures for them in advance. And as an employee of Anthropic (or Ant, as it’s sometimes known) pointed out, the company was happy to sign a contract with the Department of Defense in the first place:

Still, many believe that if you’re going to use a chatbot, Anthropic’s Claude is morally preferable to OpenAI’s ChatGPT — especially in light of the recent clash at the Pentagon.

What else can be done to ensure AI isn’t used for mass surveillance or fully autonomous weapons?

There was a time when some AI experts urged an alternative to a US-China AI arms race: What if Americans who care about AI safety tried to coordinate with their Chinese counterparts, engaging in diplomacy that could ensure a safer future for everybody?

But that was a couple of years ago — eons, in the world of AI development. It’s rarer to hear that option floated these days.

Some experts have been calling for an international treaty. A dozen Nobel laureates backed the Global Call for AI Red Lines, which was presented at the UN General Assembly last September. But so far, a multilateral agreement hasn’t materialized.

In the meantime, another option is gaining prominence: solidarity amongst the tech workers at the major AI companies.

An open letter titled “We Will Not Be Divided” has garnered more than 900 signatures from employees at OpenAI and Google over the past few days. Referring to the Pentagon, the letter says, “They’re trying to divide each company with fear that the other will give in. That strategy only works if none of us know where the others stand. This letter serves to create shared understanding and solidarity in the face of this pressure.” Specifically, the letter urges OpenAI and Google leadership to “stand together” to continue to refuse their AI systems to be used for domestic mass surveillance or fully autonomous weapons.

Another open letter — which has over 175 signatories, including founders, executives, engineers, and investors from across the US tech industry, including OpenAI employees — urges the Department of Defense to withdraw the supply chain risk designation against Anthropic and stop retaliating against American companies. It also urges Congress “to examine whether the use of these extraordinary authorities against an American technology company is appropriate” — a tactful way of suggesting, perhaps, that the Pentagon’s moves were an abuse of power. 

Federal regulations and global treaties would be a much stronger defense against unsafe and unethical AI use than relying on the goodwill of individual technologists. But for the moment, cross-company coordination is at least a start — a way to push back against Pentagon pressure that would lead, if left unchecked, to something America keeps insisting it’s nothing like.