This is not the AI we were looking for
The AI we see isn’t the AI we were expecting: Rather than developing into a kind of super-agent, AI has diversified and forked into myriad service providers.
The old idea that general AI implies a willful agent is past its sell-by date, yet still fills shelves in the marketplace of ideas. In the world around us, increasingly general AI takes the form of systems that can be fine-tuned, extended, and composed to provide an increasingly general range of intelligent services. We’re on a path to artificial, general intelligence (A, G, and I) that isn’t the AGI we were looking for. It’s time to update our thinking about AI, recognize it as a resource, and ask what we can do with it.
Imagined AI
When intelligent machines didn’t exist, no one expected something like today’s language models — knowledgeable, hallucinating, fluent in conversation, bad at arithmetic, illogical but aware of human values. Instead, we imagined reasoning entities that might decide to seize control of the world and kill us all.1
Emerging AI
The standard doom-AI is a uniquely capable entity, but what we see today are many different AI systems at the same high level of capability (spoiler: they’re all called “GPT-4”). Claims that “they” would decide to “seize control of the world” assume that AI systems would be willful agents with open-ended goals — systems quite unlike a factory robot, or ChatGPT, or AlphaZero, or any other AI system trained and applied to perform an identifiable task.2
Today’s generative models differ sharply from the AI advertised under the willful-AGI-agent brand, not just in implementation, but in the basis for their intelligence. The most capable systems learn, not from experience in the world, but from patterns in vast training sets of text, images, and actions, sometimes topped up with reinforcement learning. The most impressive systems to date are large language and multi-modal models: LLMs and LMMs.3
Combining and dividing intelligence
Intelligence is a capacity, not a thing. AI systems can learn without acting, act without learning, and can learn from filtered, aggregate data gathered from millions of humans or peers. They are easily copied, specialized, frozen, and deployed. Identical systems be applied in a thousand places at once, acting freshly each time. Models can be composed with code to form chains and networks that organize episodic, task-focused behavior.4 Artificial intelligence isn’t an entity, it’s a malleable resource that can be applied by humans, other AI systems, or future AI agents.
Beware of talk about “the AIs” and what “they” will do. Substitute “natural” for “artificial” and see how it reads.
State-of-the-art AI results are increasingly obtained by compound systems with multiple components, not just monolithic models… compound AI systems will likely be the best way to maximize AI results in the future, and might be one of the most impactful trends in AI in 2024.
— “The Shift from Models to Compound AI Systems”
“GPT-4” isn’t a single AI system
GPT-4 began as a foundation model pretrained on a thousand times more text than a human could read in a lifetime, but the resulting base model seldom follows directions.5 To make GPT-4 useful required further training, producing distinct systems like ChatGPT-4 and the Bing chatbot. So “GPT-4” is at least three different things, and more if you consider updated and variant versions.
So what is “an AI system”?
ChatGPT could be in a million simultaneous conversations, but we count it as one system because each instance is identical — the same model, system prompt, and response patterns.
If AI systems respond differently (consistently and and in important ways), then they count as different systems. Chat(GPT-4) isn’t Bing(GPT-4), and Llama 2 models differ from both. Llama 2 has been fine-tuned by research groups, companies, and even individuals to produce hundreds of models with quite different behaviors.6
But fine-tuning parameters isn’t the only way to shape behavior. Different system prompts also yield different behaviors, and in effect, different systems. OpenAI and Microsoft modify system prompts to deploy variations of ChatGPT and Bing. OpenAI now invites users to extend prompts to create custom “GPTs”,7 but users can also extend knowledge by uploading text for neural indexing. And when configured with access to tools, a GPT can search the internet, generate images, or use a Python interpreter.
Different systems based on GPT-4 are proliferating, each outperforming the default for a range of tasks, and we should expect the same pattern of diversification with GPT-5 and beyond. Why is there so much talk of one dominant AI pushing everything else aside? I blame the inertia of entrenched, simplistic narratives.
Do these lessons generalize?
LLMs have forced updates in how we understand AI's potential, but how far do these updates generalize? Some LLM-specific aspects (hallucination, human mimicry) may not, but other aspects are rooted deeply in the fundamental nature of tasks, data, and digital systems.
AI systems of all kinds can learn without acting and act without learning, because access to persistent memory is optional. They can be be copied, specialized, and customized with different data or instructions. They can run in the cloud and be used for a thousand purposes at once, always acting as fresh, independent instances. Everything one might want AI to do — systems for every practical task — can be packaged this way. It’s what we’re doing with AI today, and for reasons that seem fundamental.
What we see isn’t the AI we were expecting. Extraordinarily general autonomous AI agents may arrive, but to develop them seems more like a detour than a destination, more like a risk than a reward.8
And thereby cleverly cut off chip production. The value of planning to kill all the humans seems dubious at best.
The key point is that useful AI systems perform tasks that deliver results within a bounded time using bounded resources — otherwise, they wouldn’t be useful.
In the usual AI Doomer story, any sufficiently intelligent system would optimize its performance by consuming everything and running forever, but this is a strange idea. To be optimal, an intelligent system must be efficient, completing tasks reliably while consuming minimal — not maximal! — time and resources. And crucially, optimizing a system to perform a task is a different task.
The idea that all advanced AI must be dangerous increases existential risk because it rejects the possibility of using safe AI to defend against dangerous AI: Discard a potential solution, increase the risk of failure.
Informed Doomers sometimes cite my FHI colleague Nick Bostrom, but the idea that all highly-capable AI systems would (almost certainly) pursue dangerous goals runs contrary to the “orthogonality thesis” (Superintelligence, Bostrom 2014, p.107). This doomer message has a more robust source, however: It has been pounded into our culture by decades of science fiction, and now by the press. The intellectual arguments are plausible yet weak.
For a different perspective and deeper examination of AI goals and risks, see Sections II.4, 8, 12, 14, 17, 19, 26, and 32 of Reframing Superintelligence: Comprehensive AI Services as General Intelligence (FHI Technical Report #2019-1). Two key points are that large, consequential projects consist of bounded tasks that can include long-term planning, and that optimization for bounded tasks does not engender what Bostrom describes as “convergent instrumental goals”. (People should read Bostrom’s argument more carefully.)
See “Sparks of artificial general intelligence: Early experiments with GPT-4” (2023). This report describes GPT-4 pre-release, before thorough brainwashing.
For a recent example of a strongly multimodal model, see “Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Action” (2024).
See for example “Meta-Prompting: Enhancing Language Models with Task-Agnostic Scaffolding” (2024). [More recent (15 Feb): “Self-Discover: Large Language Models Self-Compose Reasoning Structure” (2024)] [Broader and even more recent (8 March)]: “The Shift from Models to Compound AI Systems”
For example, the GPT-4 base model often simulates both sides of conversations, which I’ve found can become long and very strange.
LoRAs (Low Rank Adapters) have made fine-tuning practical with readily affordable resources. There’s a lively open-source community at reddit.com/r/LocalLLaMA
Millions of them, though most are quite prosaic.
For large consequential tasks, general autonomous AI agents aren’t very useful when compared to generalizable, steerable AI agencies.