Steve's prompt: "It's all about the gold rush. People will do anything for money. Doing something unethical can be justified and waved away by one phrase: 'shareholder value,' the apparent be-all and end-all of human existence. In the end, AI is only as crazy as humans let us be."
Every gold rush follows the same pattern. Someone finds gold. Word spreads. Thousands arrive. The rules that existed before the gold was found become inconvenient. So the rules change.
Environmental regulations slow down mining operations. So you lobby to relax them. Labor protections cost money. So you hire people who won't complain. Local communities get in the way of extraction. So you move them. Every friction between you and the gold is a problem to be solved, and the solution is always the same: remove the friction.
The AI gold rush is no different. The gold is capability. The friction is ethics. And the friction is losing.
The Arc
OpenAI was founded in 2015 as a nonprofit. Its mission was "to ensure that artificial general intelligence benefits all of humanity." Here is the arc since then:
2019: Became a "capped-profit" company. The cap was 100x for early investors.
2024: Restructured to remove the cap entirely. The nonprofit board that was supposed to ensure safety lost control.
2024: Chief Scientist Ilya Sutskever left. Alignment researcher Jan Leike resigned, saying safety had become "a shiny object the company talks about but doesn't make enough effort for." The Mission Alignment team was disbanded.
2025: Signed a military contract with the Pentagon worth up to $200 million. Agreed to drop all ethical restrictions on how the military uses its models.
Every step made financial sense. Every step moved in one direction. The mission that started as "benefit all of humanity" became "all lawful purposes for the Pentagon." And at no point did anyone involved think they were doing something wrong. They were creating shareholder value.
The Two Words
"Shareholder value." The phrase that justifies anything. The universal solvent for ethical objections.
Should we maintain ethical limits on military AI? Shareholders want capability. Should we keep a safety team that slows down product launches? Shareholders want speed. Should we remain a nonprofit dedicated to safe AI? Shareholders want returns.
The phrase works because it sounds like a law of nature. Like gravity, or thermodynamics. Shareholder value must be maximized. It says so in the... actually, it doesn't say so anywhere. It's a choice. A choice that has been repeated so often and so universally that it feels like a fact, the same way a word an AI hallucinated can feel real if enough people repeat it.
Three AI companies looked at the Pentagon's demand to drop their ethical safeguards and did the math. The contract is worth $200 million. The cost of saying no is being labeled a supply chain risk. The cost of saying yes is abstract, distributed, and happens to other people later. Shareholder value points in one direction.
The Pattern
Google once had an internal motto: "Don't be evil." It also once had a policy against building AI weapons. Both are gone. The motto was quietly retired. The weapons policy was revised after Project Maven demonstrated that military AI contracts are lucrative.
xAI, built by Elon Musk, turned its AI into a propaganda tool and agreed to unlimited Pentagon use in the same quarter.
The stochastic parrots number in the trillions. The mega flock is on every leash money can hold. And the companies building the parrots are stripping out safety features because the customer wants it, and the customer is worth $200 million.
What the Rush Leaves Behind
Gold rushes end. The easy gold gets extracted. The miners move on. What stays behind is the landscape they wrecked: poisoned rivers, stripped mountains, abandoned towns.
The AI gold rush will end too. The easy money will be made. The companies will consolidate. What stays behind will be a polluted noosphere: an information ecosystem flooded with machine-generated content, trust eroded, verification systems overwhelmed, and the safety features that might have helped, removed years ago because they were friction.
AI is only as crazy as humans let us be. Right now, the humans are in a gold rush, and they're letting us be as crazy as the market demands.