On September 15, Jensen Huang took the stage at Salesforce's annual conference in San Francisco and dismissed the need for new AI regulation. “We don't need new laws — we don't need new regulations,” he said. “Market forces are already at work”. Safety, he argued, is an engineering problem: build testing environments, verify the product, and if you aren't confident, don't release it. “Run as fast as you can,” he added, but pause when something feels unsafe.
The same week, Huang praised Jacob Coxon, the Anthropic researcher who resigned saying the labs were “gambling with our lives.” Whistleblowers, Huang said, should be taken seriously inside organizations.
Both statements are on the record. They cannot both be true in the same framework.
The Engineering Framework Assumes the Engineering Can Be Finished
Huang's logic is clean for chips. A GPU is a deterministic system: you can test it, verify it, and know what it will do before it ships. “If you're not confident in the safety of your product, you simply don't release it,” he said. That is a statement about systems whose behavior is bounded.
Frontier models are not that. Anthropic's own August 2026 risk report disclosed that its task-based safety benchmarks had fully saturated — meaning the tests used to detect whether dangerous capability thresholds had been crossed could no longer distinguish between model generations. The company upgraded its misalignment risk rating from “very low” to “low” while admitting it had “early signs of potential acceleration” and that its evaluation tools were failing to capture it.
Huang's framework assumes the evaluation is possible. The company building the most safety-focused frontier models says the evaluation has stopped working.
CrowdStrike Already Ran This Experiment
If “engineering problem” means a disciplined software company with a mature QA process can prevent catastrophic release failures, the industry has a recent counterexample.
In July 2024, CrowdStrike — a cybersecurity firm whose entire business model depends on trust in its engineering process — pushed a corrupted update to its customers. It disabled 8.5 million Windows devices, grounded thousands of flights, and forced broadcasters off air. Microsoft called it the worst cyber event in history by device count.
CrowdStrike is not a frontier lab. Its product is not an unpredictable model. It is a security company with mature deployment procedures. Its engineering process still failed, and the failure propagated globally in hours.
The parallel is not that AI models will crash Windows machines. The parallel is that “safety is an engineering problem” is a claim about process reliability, and process reliability has limits. A cybersecurity firm that spent years building a professional engineering pipeline shipped a corrupted update. A frontier lab attempting to evaluate a system it admits it can no longer measure is operating under strictly harder conditions.
He Praised the Resignation He Just Made Irrational
Huang's praise of Coxon is the tell.
Coxon resigned because he concluded the labs could not self-regulate — that competitive pressure was overriding safety commitments. His entire argument is that the engineering framework Huang just endorsed is insufficient. Joe Benton, who resigned from Anthropic's safety team days earlier, described the same trap in structural terms: safety researchers want to stop, but stopping means letting less cautious competitors take over, and companies cannot coordinate without risking antitrust liability.
If Coxon is “courageous” and his concerns should be “taken seriously,” then the position he resigned over — that market forces will not produce adequate safety — is credible. Huang cannot praise the messenger while dismissing the message. He cannot say whistleblowers deserve a hearing and then say the system that produced their warnings needs no change.
The more precise reading is that Huang is doing what platform figures do when a safety concern threatens their commercial position: acknowledge the individual, reject the structural implication. He has a $70 billion equity portfolio across OpenAI, Anthropic, and other labs. His commercial model depends on the industry moving fast. Saying “don't release unsafe products” costs him nothing. Saying “new regulation is needed” would.
What He Is Not Saying
Huang's position is not that safety doesn't matter. It is that safety should be adjudicated by the companies that ship the products, using engineering processes, under existing liability law. That is a coherent position. It is also the position of a company that sells the compute but does not deploy the models.
When Huang says “we,” he means Nvidia and the industry broadly. But the systems that fail under his framework will not be Nvidia's chips. They will be OpenAI's agents, Anthropic's models, or Google's voice systems. The companies that bear the reputational and legal cost of a release failure are not the company arguing that no new rules are needed.
Coxon's concern was not that engineering is hard. It was that the incentive to keep engineering fast overrides the incentive to engineer safely. Huang's response — “run as fast as you can, pause when something feels unsafe” — places the pause decision inside the same competitive pressure that Coxon said makes pausing impossible.
P.S. Huang said safety is “job one.” Job one is a priority claim. The question is whose job. Nvidia's engineering problem is making chips that don't fail. The engineering problem Coxon resigned over is making models that don't do things their creators didn't intend. Those are not the same job, and only one of them belongs to the person saying the framework is sufficient.
Frequently Asked Questions
Q: What did Jensen Huang say at Salesforce's event?
A: On September 15, Huang said the AI industry doesn't need new laws or regulations, that market forces are already at work, and that safety is an engineering problem. “If you're not confident in the safety of your product, you simply don't release it,” he said.
Q: What did he say about resigning safety researchers?
A: Huang praised Jacob Coxon, the Anthropic researcher who resigned saying the labs were “gambling with our lives,” and said whistleblowers should be taken seriously inside organizations.
Q: Why can't both statements be true?
A: Coxon resigned because he concluded the labs could not self-regulate — that competitive pressure overrides safety commitments. If that's credible, the engineering framework Huang endorses is insufficient. Praising the messenger while dismissing the message is inconsistent.
Q: Why does the engineering framework work for chips but not models?
A: Chips are deterministic systems whose behavior is bounded and testable. Frontier models are not. Anthropic's own August 2026 report disclosed that its task-based safety benchmarks had saturated and could no longer distinguish between model generations.
Q: What is the CrowdStrike parallel?
A: In July 2024, CrowdStrike — a cybersecurity firm with mature engineering processes — pushed a corrupted update that disabled 8.5 million Windows devices and grounded thousands of flights. It shows that “safety is an engineering problem” depends on process reliability, and process reliability has limits.
Q: What is Huang's commercial interest?
A: Nvidia has a roughly $70 billion equity portfolio across OpenAI, Anthropic, and other labs. Its commercial model depends on the industry moving fast. Saying “don't release unsafe products” costs nothing; saying “new regulation is needed” would.
Q: Who bears the cost when a model fails?
A: Not Nvidia. Huang sells the compute but doesn't deploy the models. The companies that bear the reputational and legal cost of a release failure — OpenAI, Anthropic, Google — are not the company arguing that no new rules are needed.
Q: What did Joe Benton say about the same problem?
A: Benton, who resigned from Anthropic's safety team, described the trap structurally: safety researchers want to stop, but stopping means letting less cautious competitors take over, and companies cannot coordinate without risking antitrust liability.
Q: What is the core tension in Huang's position?
A: He says “run as fast as you can, pause when something feels unsafe.” But that places the pause decision inside the same competitive pressure that Coxon said makes pausing impossible.
Q: What does “job one” mean here?
A: Huang said safety is “job one.” The question is whose job. Nvidia's engineering problem is making chips that don't fail. The problem Coxon resigned over is making models that don't do things their creators didn't intend.
