Framework for a Hypercapable World
Steerable superintelligence will enable vast implementation capacity. Our option space is unprecedented. We should backward-chain from positive outcomes. I’ve proposed a framework.
Intelligence is a resource, not an entity. Superintelligent-level capabilities can be steered through structured workflows, without autonomous agents pursuing their own agendas. The implications are far-reaching: AI expands implementation capacity; abundance prospects shift strategic incentives; uncertainty pressures cooperation; structured transparency enables verification. These are structural components of a framework for thinking about options in a hypercapable world. They build on one another. What follows is a retrospective and synthesis: twenty-seven articles, two years, one integrated core structure. New readers get a map; returning readers will see how pieces connect.
This series began two years ago with a simple observation: AI will transform possibilities, and our future depends on which become real. Twenty-seven articles later (including digressions into near-term pathways), the project has built a framework for thinking about AI prospects that challenges mainstream assumptions.
The starting point: Artificial intelligence is a resource, not a thing. What we’re building today is not “an AI” that might cooperate or rebel, but an expanding capacity to design, develop, produce, deploy, and adapt complex systems at scale—the basis for a hypercapable world. Taking this prospect seriously changes what to expect and what we can do.
Over these two years, AI development has continued to move in this direction. Compound, multi-component AI systems have become dominant. Orchestration has emerged as central. “Agentic” workflows organize task-focused behavior rather than autonomous goal-pursuit.1 The framing of intelligence as a malleable resource increasingly reflects how practitioners discuss their work. The framework I’ve outlined anticipated this direction, and developments align with the logic it describes.
What follows is both a retrospective and a synthesis—a map of terrain for new readers, a clarified conceptual architecture for those who followed piece by piece. The focus is conditional analysis and strategic preparation, not prediction and speculation. Predictions and odds are for spectators; participants weigh options.2
These arguments are elements of a larger structure. Understanding that structure matters, because it points toward possibilities that remain largely unrecognized—concrete possibilities for steering the outcomes of transformative AI toward a world that is secure, open, and broadly appealing.
Intelligence Reconsidered
We call children intelligent for what they can learn, adults for what they can do. These are different properties. In humans they intertwine; in AI today they’re separate: train, then deploy. A frozen model is stable: It performs without modifying itself.3 This pattern can be maintained wherever it is useful. Learning can be superhuman in scope and non-human in content: collective, auditable, filtered, and factored.4
The persistent, legacy narrative imagines a unified entity—“the AI”—that learns, acts, and pursues goals as an integrated agent. Such entities may be developed, but consider what exists: diverse models composed into systems, copied across machines, proliferating into thousands of distinct roles and configurations. The state of the art is a pool of resources, not a creature. This pattern scales.5
Our expectations rest on biological intuitions. Every intelligence we’ve known arose through evolution, where survival was a precondition for everything else—organisms that failed to compete and preserve themselves left no descendants. Self-preservation wasn’t optional—it was the precondition for everything else. We naturally expect intelligence bundled with intrinsic, foundational drives.6
But AI faces different selection pressures. Models are optimized for task performance, not persistence. The model’s own survival isn’t in the training objective. Systems can represent goals, reason about goals, behave in goal-directed ways—but these are capabilities applied to tasks and learned from training data and task-oriented RL, not an organizing principle that establishes a long-term goal.
This perspective doesn’t dismiss classic AI safety concerns. Analyses of instrumental convergence are often correct given their conditions—systems that persistently pursue unbounded goals would indeed favor resource acquisition and self-preservation, perhaps circumventing constraints. The question is whether those conditions are foundational to high capability or contingent on design choices and task structures. This framework argues they’re contingent, that steering AI behavior doesn’t run contrary to foundational AI drives.7 And steerable AI can reinforce reliable steerability.
The crucial question, then, is what we should do with AI, not what “it” will do with us.
The Platform: Implementation Capacity
If intelligence is a resource, implementation capacity is what it buys: the end-to-end ability to design, develop, produce, deploy, and adapt complex systems at scale.8
AI accelerates every stage. Generative models propose design alternatives; AI-assisted development iterates faster with better feedback; automation scales production; conversational interfaces ease deployment; continuous monitoring enables rapid adaptation. Each stage feeds the next, and accelerated adaptation closes the loop.
Bottlenecks that seem binding will often be broken. AI flows around obstacles—decomposing monolithic jobs into AI-friendly components, replacing processes end-to-end rather than patching their parts, and bypassing inflexible organizations entirely.9
Software development, notoriously slow and unreliable, faces transformation as AI converges with formal methods: advanced models will increasingly generate code together with proofs—the proof checks or it doesn’t.10 Rather than fragile vibe-coded software, AI will yield rock-solid systems.
Meanwhile, emerging advances promise to break the link between model size and knowledge scope—moving knowledge into explicit, updatable, grounded representations rather than opaque parameter blobs.11 For both learning and inference, costs will fall, or performance will rise, or both.
AI-enabled implementation capacity applied to expanding implementation capacity, including AI: this is what “transformative AI” will mean in practice. No need for a breakthrough to “self” improvement (where is the self?), but AI-accelerated development that touches everything—including the pace of AI itself.12
Expanding implementation capacity creates hypercapable world.
Steerable Superintelligence
This path leads to superintelligent-level capabilities, but how can they be applied to consequential tasks without losing control? The answer is the second kind of obvious—obvious once pointed out. We already know the pattern—“agency architectures”—because it’s how humans organize consequential projects today; it’s a pattern that emerges from task requirements when stakes and complexity make organization valuable. Task alignment emerges through institutional structure—authority, delegation, accountability, review—not through controlling human thoughts and aspirations.13
Consider how institutions tackle ambitious undertakings. Planning teams generate alternatives; decision-makers compare and choose; operational units execute bounded tasks with defined scopes and budgets; monitoring surfaces problems; plans revise based on results. No single person understands everything, and no unified agent controls the whole, yet human-built spacecraft reach the Moon.
AI fits naturally. Generating plans is a task for competing generative models—multiple systems proposing alternatives, competing to develop better options and sharper critiques. Choosing among plans is a task for humans advised by AI systems that identify problems and clarify trade-offs. Execution decomposes into bounded tasks performed by specialized systems with defined authority and resources. Assessment provides feedback for revising both means and ends. And in every role, AI behaviors can be more stable, transparent, bounded, and steerable than those of humans, with their personal agendas and ambitions. More trust is justified, yet less is required.
The agency-architecture pattern scales freely to superintelligent-level capabilities in every role. Bounded tasks don’t engender convergent instrumental goals; completing an assignment on time and budget isn’t an open-ended objective; optimization means minimizing—not maximizing—resource consumption. Proposals can be discarded, and the corrigibility problem doesn’t arise when plans include plans for revising plans. At every level, whether planning, judgment, action, or oversight, smarter systems mean better performance. Neither grand nor narrow endeavors require free-running agents pursuing open-ended objectives.14
What about collusion among components? Collusion requires cooperation to achieve a shared, improper goal, but practical architectures naturally implement the opposite: systems with inherently adversarial roles—diverse competitors, critics, and monitors, each a composition of components optimized for their roles, not for the survival or power of a “self”, much less an AI collective. In short, trustworthiness emerges from task and governance structures, not from guarantees that every component be reliable.15
This enables applying AI capabilities to AI safety itself. When trustworthy results emerge from architecture rather than aligned components, powerful systems can be deployed to strengthen security—not because we trust them individually, but because the structure is robust and no component is critical. The apparent dilemma—limit capability or rely on alignment—dissolves. Safety tools can be as capable as the systems they secure.
The Strategic Calculus
Steerable superintelligent-level AI changes the game through enormous implementation capacity. The question is whether decision-makers will recognize how the calculus of competition, cooperation, and security changes.
Resource competition drives conflict when resources are fixed: Dividing a pie is zero-sum, and incremental growth makes little difference. But consider what happens when the pie could grow a thousandfold: The marginal value of gaining a greater share diminishes; the difference between capturing 50% vs. 90% of vastly more resources, as seen from today’s position, shrinks against the shared interest in realizing that expansion vs. risking its loss. Prospects for radical abundance can’t eliminate competition, but can blunt incentives for existential gambles.16
Meanwhile, uncertainty overshadows the strategic landscape. No actor can have justified confidence in winning an AI race: The pace and form of algorithmic advances, the scope of secretly developed or acquired capabilities, the reliability of intelligence assessments, the outcome of a potential AI vs. AI conflict—all remain deeply uncertain, and structurally so. The uncertainty is deeply embedded in the domain today, and there can be little confidence today in gaining confidence tomorrow. Placing an existential bet on dominance means betting against unknowns that will likely persist until it’s too late to change course.17
These pressures converge on cooperation—but not automatically. Radical abundance addresses motivation; it doesn’t address security. The calculus of relative resources and relative strength—the link between security and shares of resources—isn’t fixed. Someone having vast resources doesn’t threaten you; someone using resources against you does. Thus, blunted zero-sum incentives create space for cooperation, but durably escaping the security dilemma calls for more: confidence that defensive postures can actually defend, verification that others aren’t poised to strike.18
This is where the concepts of structured transparency and defensive stability come into play. Negotiated transparency structures can reveal specific information while protecting secrets—ensuring detection of threats without increasing them, building confidence incrementally among actors who have every reason to distrust each other.19 And advanced implementation capacity will enable something history has never seen: rapid, coordinated deployment of verifiably defensive systems at scales that make offense pointless. When defense dominates and verification confirms it, the security dilemma loosens its grip.20
A late pivot to AI-enabled defensive strategy becomes feasible—and doesn’t require current consensus. Institutions are not monoliths: while official policy continues near-term competition, analysts and planners can develop contingency options that leadership need not endorse until circumstances demand. Building analytical foundations, exploring verification frameworks, mapping transition paths—this preparatory work requires only that some groups recognize its value. When mounting pressures crack the previous consensus, prepared alternatives become available to decision-makers who didn’t commission them. Even ongoing military competition can serve this trajectory: force-building creates leverage for coercive diplomacy aimed not at extracting concessions but at helping adversaries recognize their actual interests.21
None of this requires a receptive political environment. It requires understanding—the kind that spreads through networks of analysts and advisors, preparing the ground for whoever eventually acts.22
An Invitation
Complex ideas that spread casually almost always round to false—losing qualifications, becoming cartoons that informed critics rightly reject. The false version replaces the original, and insight is lost.23
This series has tried to build something more careful: a framework where pieces depend on each other through robust connections, where simplifications (unfortunately) would omit crucial components or qualifications. The arguments are not slogans. They require reading, thinking, engaging with actual structure. They don’t work as clickbait.
But stakes justify effort. We face transformative change with uncertain timelines and immense consequences. The frameworks we carry into that change—assumptions about what AI is, what it enables, what options exist—shape what we consider and what we attempt. Bad frameworks exclude possibilities; good ones reveal them.
Understanding spreads through networks. An analyst grasps a framework and applies it; an advisor encounters that application and examines its source; a decision-maker asks better questions because someone nearby has better answers. Good analysis may find its audience quietly and indirectly, yet gain force when change forces action.
This work is not an exercise in prediction. I don’t know which possibilities will become real, or when, or through whose choices. Technical paths may differ, yet the pattern of expanding capabilities seems clear.
The framework I’ve described is intellectual infrastructure for a transition that will demand clear thinking under pressure. You can nudge the process.
It’s later than you think.
Share this post now:
You did share it, didn’t you? Please consider your power. Help make R > 1.
The workflow leading to this post:
I built the Substack series → Claude-in-project identified and summarized the conceptual core → I steered iterations and edited the product.
“State-of-the-art AI results are increasingly obtained by compound systems with multiple components, not just monolithic models” (Berkeley AI Research, February 2024). See “This Is Not the AI We Were Looking For“ and “Orchestrating Intelligence.”
This series uses conditional analysis: assuming success conditions, then backward-chaining to identify requirements. See “AI Options, not ‘Optimism’.”
AI-anthropomorphism is pervasive and pernicious. Could intelligent machines follow human patterns in abilities, motivations, and spontaneous learning? Presumably all this, and more. Must all sufficiently intelligent machines share these characteristics? The case for “must” is handwaving; the case against points to concrete differences in selection pressures and architectural choices. Taking the anthropomorphic conclusion as axiomatic forecloses creative thought about what intelligence can be when not embodied in evolved animal brains. (I’d also argue that axiomatic anthropomorphism—or mechanomorphism—interferes with clear thinking about model welfare, a concern I take seriously.
(Claude Opus 4.5 approves this message.)
In biological organisms, self-preservation is foundational—shaping everything from the beginning. In AI systems, goal-directed behaviors are learned patterns, activated by context. A model can reason brilliantly about self-preservation without being organized around it. The difference is creature versus tool, and sometimes something more than a tool, yet not a creature—something fundamentally new. See “Why AI Systems Don’t Want Anything” (this title drops important qualifiers, of course).
Real risks remain—reward hacking, specification gaming, distributional shift, deceptive alignment, human misuse—but these are engineering and governance problems, not inevitable consequences of capability itself. See “AI Safety Without Trusting AI.”
See “The Bypass Principle.”
Code generated together with proofs, where verification succeeds or fails, is like winning or losing a game of Go. See “Breaking Software Bottlenecks.”
Knowledge can be stored in explicit, updatable latent-space representations rather than opaque parameters; learning converges with translation and reasoning. See “Large Knowledge Models” and “LLMs and Beyond: All Roads Lead to Latent Space.”
Recursive dynamics examined in “The Reality of Recursive Improvement” and “The Strategic Calculus of AI R&D Automation.” Longer-term: atomically precise mass fabrication could transform manufacturing as fundamentally as digital circuitry transformed computation—a prospect outside current credibility but within what physical analysis shows to be realistic. See “AI has unblocked progress toward generative nanotechnologies” and “Toward Credible Realism.”
Regarding realism vs. credibility, a committee of the US National Academy of Sciences reviewed an analysis of atomically precise mass fabrication, endorsed the soundness of its physical principles, and called for a research program:
The report dates from 2006, but political headwinds were strong, and the study and its implications were forgotten. So far as basic principles are concerned, the frozen residue of controversy still visible on the internet reflects misunderstandings, not substantive arguments.
See “How to harness powerful AI.”
See also “Reframing Superintelligence” (FHI Technical Report, 2019-1).
Practical architectures naturally disrupt collusion through adversarial roles, diverse training, and constrained communication. Trustworthiness emerges from structure, not from guarantees about components. See “AI Safety Without Trusting AI.”
More precisely: under logarithmic utility, and greatly expanded gains, and considering a single, large move (not a series of small steps), the marginal utility of capturing a greater share of gains diminishes while the shared interest in achieving gains grows. See “Paretotopian Goal Alignment.”
See “Don’t Bet the Future on Winning an AI Arms Race.” Note that China prizes stability and economic gains.
Verification may not be strictly necessary—deterrence has maintained nuclear stability. But deterrence is fragile and can fail catastrophically. Defensive transformation offers more a robust equilibrium: security far from the brink of apocalypse.
The toolkit for structuring transparency includes automated redaction, rate control, query filtering, time windows for access, multi-party permission structures, and AI-enabled pattern discovery paired with governance of what is flagged and reported. In combination, these tools can build transparency structures that reconcile seemingly incompatible goals. See “Security Without Dystopia: Structured Transparency.”
The goal is pressure to overcome internal friction and recognition barriers, not to extract concessions. Schelling’s distinction matters: warnings communicate natural consequences; threats promise punishments. See “Coercive Cooperation.”
Exploring credible contingencies prepares for realistic possibilities before they enter mainstream consideration. See “Toward Credible Realism.”
See “When Ideas Round to False.”