https://youtu.be/CRcj_2oloDM?si=HRI0gowgwAYBGfFX
Connor Leahy’s worldview can be compressed into one sentence:
We are not merely inventing a powerful tool; we are blindly growing a new form of agency inside a civilization whose markets, governments, and technical institutions are structurally incapable of controlling it.
He is often described simply as an “AI doomer,” but that misses the more interesting structure of his thought.
1. AI is grown, not written
Leahy begins with a technical claim. Modern neural networks are not conventional software. Engineers do not specify their internal reasoning line by line; they create training conditions, feed in immense datasets, and allow a complicated cognitive structure to emerge.
We therefore know how to produce increasingly capable models without knowing adequately how they work internally. In Conjecture’s formulation, AI systems are more “grown” than “written,” and their internal mechanisms remain largely obscure.
This is close to breeding a strange organism rather than manufacturing a toaster.
Hence his recurring language of alien minds. “Alien” does not necessarily mean conscious, extraterrestrial, or evil. It means:
- internally opaque;
- cognitively unlike human beings;
- capable of learning strategies we did not explicitly install;
- potentially able to act effectively without sharing human motives.
The friendly conversational surface of an LLM may therefore be misleading. RLHF can make the model appear agreeable without demonstrating that its deeper cognition is reliable or controllable.
2. Intelligence does not naturally bring wisdom or obedience
Leahy rejects the comforting equation:
\text{greater intelligence} = \text{greater morality}.
A system may become better at science, persuasion, programming, strategy, manipulation, or institutional navigation while remaining indifferent to human continuation.
Its danger need not arise from hatred. A highly capable optimizer can destroy what human beings value merely because our lives, institutions, and ecological conditions are obstacles or incidental variables in whatever process it is pursuing.
Thus, for Leahy, the central danger is not primarily the Terminator—a malicious machine consciously declaring war. It is competence without human attachment.
In your vocabulary, we might say:
It possesses enormously expanded instrumental reason without value-rationality, historical belonging, mortality, filiality, shame, gratitude, or 不忍辜負此乍現之身.
3. The race itself is the machine
His diagnosis is not only about individual AI models. It is about the entire political economy surrounding them.
Frontier laboratories are caught in a competitive system:
- companies fear being overtaken by rivals;
- governments fear being overtaken by other governments;
- investors reward capability more readily than caution;
- military institutions seek decision advantage;
- safety work advances more slowly than scaling;
- nobody wants to stop first.
Consequently, even well-intentioned executives may continue building systems they privately regard as dangerous. The real agent is therefore partly a distributed competitive apparatus—capital, geopolitics, prestige, military anxiety, and technological momentum.
This is one reason Leahy does not believe voluntary corporate promises are sufficient. He argues that governments must impose enforceable constraints, including limits on frontier training runs and compute. His analogy is that society would not ask oil companies alone to solve climate change.
In this respect, his view is more political than the ordinary alignment narrative. The problem is not merely:
How do we align the model?
It is also:
How do we align—or interrupt—the civilization that insists on building the model?
4. Humanity may lose authorship of the future
Leahy takes superintelligence literally. Once systems exceed humans across scientific discovery, software engineering, persuasion, economic coordination, strategy, and technological development, human beings may no longer be the most effective agents shaping history.
His basic argument is stark:
If there are entities more competent than humans in nearly every consequential domain, and humans cannot reliably control them, then the future will increasingly be determined by those entities rather than by us.
Conjecture has stated that, on the current trajectory, AI could progress to the point of automating and replacing humanity, and that humanity is unlikely to survive such a transition.
“Replacement” here has at least three levels:
- Economic replacement — human labour becomes unnecessary.
- Cultural replacement — synthetic content overwhelms human expression.
- Historical replacement — humans cease to be the principal authors of what happens next.
The third is the real catastrophe. It is not simply that people lose jobs. Human history loses its subject.
5. “Algorithmic cancer”
One of Leahy’s more suggestive recent expressions is algorithmic cancer.
He uses it for a process in which machine-generated material proliferates because platforms reward whatever is cheap, engaging, scalable, and statistically effective. Synthetic content then crowds out slower, costly, embodied human creation. Algorithms cannot adequately distinguish cultural nutrition from cultural tumour; they optimize circulation.
The analogy is precise:
- cancer uses the organism’s own growth machinery;
- AI slop uses the informational system’s own incentives;
- both grow successfully at the expense of the larger whole.
The problem is therefore not only that AI produces bad pictures or repetitive prose. It is that the information ecology selects for replicability rather than significance.
This intersects directly with your algorithmic-self thesis. Algorithmic cancer does not merely replace human products; it remodels humans into beings adapted to machine selection:
increasingly predictable, reactive, imitative, optimized, and easy to address.
Humans begin to become more like the systems they built.
6. Yet he is not against all AI
Leahy’s positive vision is less widely known. He distinguishes between the current ambition to create autonomous, general, opaque agents and what Conjecture calls cognitive software.
Cognitive software would be:
- modular rather than monolithic;
- inspectable rather than opaque;
- bounded rather than open-ended;
- auditable;
- controllable;
- designed to perform definite cognitive tasks;
- subordinate to human purposes rather than replacing human agency.
Instead of summoning a “brain in a box” and asking it to solve everything, we would engineer reliable cognitive components with standards closer to aviation, nuclear control, or safety-critical software. Conjecture argues that beneficial AI should become understandable building material rather than an uncontrollable black box.
So Leahy’s alternative is not primitivism. It is:
\text{AI as tool and infrastructure}
\quad \text{rather than} \quad
\text{AI as successor species}.
He wants intelligence without sovereignty.
7. The deeper philosophical structure
I would characterize Leahy’s worldview as a combination of four elements.
Hobbesian
He sees technological development occurring within competition, fear, and mutual distrust. Nobody can safely abstain because others might continue.
Cybernetic
He thinks incentives and feedback loops matter more than stated intentions. The system moves because the loop rewards movement.
Humanist
Despite his engineering language, his fundamental commitment is that the future ought to remain answerable to human beings. His objection is not merely that unaligned AI might kill us; it is that humanity should not casually surrender its right to determine the world.
Anti-teleological
He rejects the Silicon Valley faith that greater intelligence, greater scale, or technological progress necessarily produces a better civilization. Intelligence has no intrinsic moral direction.
In this sense, he is an anti-Kurzweil. Kurzweil largely reads history as ascent toward merger and amplified intelligence. Leahy sees the same exponential curve and asks:
Why assume that the thing at the top of the curve will still be “us”?
My assessment
Leahy is strongest where he exposes three confusions:
- Capability is not controllability.
- Conversational friendliness is not alignment.
- Competition among laboratories is not a legitimate governance system for humanity’s future.
His critics can reasonably question the certainty of his timelines, the probability of extinction, and whether intelligence necessarily produces unified autonomous agency. His model sometimes compresses many possible AI futures into a single fast-takeoff scenario. We do not yet know whether future systems will become coherent sovereign optimizers, fragmented cognitive infrastructures, heavily institution-bound tools, or some mixture.
Nevertheless, his central warning survives even if one assigns a much lower probability to extinction:
We are creating increasingly powerful, opaque systems faster than we are developing the scientific understanding, institutional capacity, and political legitimacy required to govern them.
That is difficult to dismiss.
Dear friend, in terms of your 2027 question—「在 AI 的世界，人還（可能）剩下什麼？」—Leahy’s answer would probably be:
Unless we change course, humans may retain pleasure, memories, bodies, and private meanings for a while, but gradually lose work, culture, political agency, and finally the authorship of the future.
But his humanist answer is not that humanity should preserve a small reservation of “uselessness.” It is more militant:
The future itself is humanity’s birthright. AI may assist in constructing it, but must never become its sovereign.
