Large Knowledge Models

Navigating the AI transition with AI assistance: Deeply integrated knowledge as a target for differential acceleration

Advanced AI will transform possibilities, and our future will depend on which possibilities become real. I am persuaded that our options are both narrower and broader than they may seem, and that if understood, they would tend to align a range of seemingly clashing interests.¹ A critical challenge is the enormous gap between emerging capabilities and our collective understanding of their implications.

In a world informed, as it has been, by a stream of policy papers, conferences, news reports, and tweets, can we expect a breakthrough in understanding fast enough to cope with accelerating, transformative change? This, to put it mildly, seems unlikely. The weakness of our knowledge ecosystem leaves us blind to both catastrophic risks and transformative opportunities.²

If technological capabilities will advance at AI-driven speed, then our understanding must somehow keep pace. This article explores how AI-enabled knowledge integration can help.

Visitors from the recent past would likely see advanced LLM technologies as indistinguishable from magic. What if we could extend today’s information-magic with a few crucial improvements?

AI Knowledge-Magic™ for a Wonderfully Informed World™

Let's try to imagine a Wonderfully Informed World™ made possible by applying AI Knowledge-Magic™ to integrate information at a civilization scale. What might that be like?

AI provides Large Knowledge Models (LKMs)

In our wonderfully informed world, AI systems have gathered and digested all publicly available information and made it universally accessible. The state of knowledge, ignorance, and controversy has become transparent in areas like science, technology, economics, and the world situation as a whole.³

Knowledge in LKMs is grounded in external sources. Cumulative reasoning extends knowledge with confident inferences, and this reasoning can be examined and traced to its sources. Misconceptions and disinformation become easier to identify and reject.⁴

Better integration and synthesis of information supports general implementation capacity — the ability to design, develop, deploy, apply, and adapt complex systems at scale.⁵ Building workable plans requires deep knowledge of diverse domains, knowledge that LKMs can provide.

Knowledge models are tools for expanding human capabilities, not autonomous entities. Knowledge modeling illustrates how intelligence can serve as a malleable resource, a perspective critical for developing realistic AI safety strategies.⁶

Large Knowledge Models reveal technological possibilities

Technological possibilities — in a fundamental physical and computational sense — are facts about the potential of the world. They include how matter and energy can be transformed into useful products, and what those products could do. Impossibilities, too, are facts, because technological possibilities are subject to objective constraints.

Consider atomically precise mass fabrication (APMF): Few informed experts⁷ doubt its physical possibility, yet its prospects remain obscured by outdated controversies and fragmented information.⁸ Knowledge models could synthesize and clarify the relevant information, helping bridge the gap between what is credible (strongly automated production) and what is realistic (fully automated, physics-limited production).

Fragmented knowledge also obscures prospects for structured transparency frameworks as a basis for powerful policy options. Current discussions of privacy, security, and surveillance neglect novel options that could greatly reduce tradeoffs between security and privacy. Understanding these options, however, would require a synthesis of knowledge from multiple domains.⁹

Understanding possibilities requires modeling how capabilities can combine to form higher-level capabilities. Knowledge gaps can be extraordinarily costly, because ignorance of crucial possibilities undercuts awareness of all the possibilities they enable. And if those possibilities include solutions to problems of existential magnitude, then perhaps better knowledge integration should be a first-rank priority.¹⁰

Large Knowledge Models facilitate reasoning about policy options

Potential policies are constrained by facts, and many key facts will (in a wonderfully informed world) be clear. Some are facts about technological possibilities, some are facts about clashes and alignments of interests, some are facts about uncertainties about other facts.

Human and machine creativity can build knowledge structures around policy options that reflect technological possibilities and emergent win-win options. This understanding is crucial for preparing to pivot — developing, deepening, and explaining complex strategic options when mounting pressures demand change.

Technology-enabled possibilities include:

• Win-win material abundance, avoiding economic conflict

• Win-win defensive stability, avoiding military conflict

• Rapid yet incremental transitions from offensive to defensive force postures

As technological possibilities become better understood, more of what is realistic will become credible, allowing us to move beyond current deadlocks in strategic thinking.

Large Knowledge Models help save the world

Above I asked whether, in a world informed by policy papers, conferences, news reports, and tweets, we can expect a breakthrough in understanding, with the obvious negative response. Now let's ask, "In a world informed by Large Knowledge Models, could we reasonably hope for better outcomes?"

There are pros and cons, because the implications of better knowledge are unpredictable. On the whole, however, I'd bet on better knowledge leading to better outcomes. As perceived options expand and become better understood, opportunities for goal alignment emerge. Even actors with competing interests may find common ground in pursuing outcomes that benefit all parties.¹¹ In a world of potentially unstable military competition, goal alignment may be crucial.

Reducing AI Knowledge-Magic™ to practice

Is it reasonable to expect that AI can be leveraged to build knowledge-model functionality of the kind suggested above? The answer, I think, is an unambiguous “Yes”.

LLMs demonstrate that AI systems can ingest vast amounts of knowledge, learn reasoning skills, and can use their knowledge and reasoning capacity to guide search and help interpret external information. Relying on external information — “retrieval augmented generation” (RAG) — reduces reliance on LLMs' faulty, opaque, and too-creative memories. OpenAI’s “Deep Research” illustrates this well, producing credible research reports based on iterative reasoning and internet search.

The knowledge model concept proposes to add cumulative synthesis and refinement of that information — upgrading quality by filtering out semantic noise and seeking coherence guided by provenance, consensus, and especially consilience.¹² Knowledge can be extended with explicitly grounded reasoning, guided toward fruitful results by LLMs' ability to “synthesize across complex subjects”.¹³ From an LLM perspective, an LKM would enable high-quality, latent-space RAG; from an external perspective, the combination would act as AI system with scalable long-term memory.

Don't expect a single, dominant knowledge model, or excellent systems without precursors. As with LLMs, there will be competing versions and incremental upgrades, and as with LLMs augmented by RAG, there will be many information stores, some private and proprietary. This diversity is a feature, not a bug — it prevents single points of epistemic failure while still fostering convergence on shared understandings of reality.¹⁴

Some theoretical and practical considerations

All roads lead to latent space

Knowledge can best be represented and synthesized in a language-independent semantic space ("latent space"), not by accumulating strings of words in English, Chinese, or Esperanto. Language-oriented ML practitioners will recognize the enormous, qualitative advantages of working with sets of differentiable dense-vector representations ("embeddings" of meaning in a high-dimensional semantic space) rather than token sequences.¹⁵ Embeddings are strictly more expressive than text.

ML practitioners with ample time to follow the literature may be aware of recent advances in compressing text-space information into denser Transformer-compatible latent-space representations,¹⁶ and in using latent-space representations as a medium for reasoning. There is much more to be said about how the pieces can fit together at a technical level.

Current language models must reconstruct meaning from text each time they access it. Combining and re-expressing knowledge as text incurs not only computational overhead but also a kind of semantic quantization noise that degrades meaning.¹⁷ Persistent knowledge structures in latent space could enable cumulative refinement and combination of complex ideas, representing clashes, agreement, nuance, analogy, uncertainty, and explicit ambiguity in machine-native ways that natural language cannot.

Grounding, updating, and scaling

LLMs suffer from hallucinations, and their internal representations can’t be annotated with source citations. Large knowledge models would retain the benefits of latent-space representations (the substance of Transformer processing) but in the form of distinct data objects that can be annotated with sources.

This same structural advantage allows knowledge to be added, removed, and updates without interfering with existing knowledge or requiring retraining. Affordances for tracing knowledge sources and derivations can make reasoning more interpretable.

LLM training shows that large models can ingest information at a civilizational scale and at a large but affordable cost. Transformer-based models with broadly comparable costs can translate text into compressed, Transformer-native vector representations. Vector databases — implementing a kind of associative memory — can scale to billions of information units and beyond. Extensive reasoning to upgrade and synthesize knowledge is affordable, and can largely be driven by demand, like other forms of LLM inference.

Epistemics

The term "knowledge model", like most short sequences of words, is easily misunderstood. If "knowledge" is taken to mean "truth", then the conversation runs off the rails to rehash (here imagine the usual disputes regarding the meaning of truth, mistaken beliefs, legitimate controversies, social construction, and so on, ad nauseam).

"Knowledge models" (inevitably plural) could present specific views of what is true, but to build shared, trusted resources, it is natural to adopt a different epistemic stance, a perspective that is in a sense neutral to questions of truth, yet presents coherent, well-grounded perspectives to anyone seeking a reality-aligned understanding of the world. Presenting perspectives would typically be a task for LKM-informed upgrades to the (somewhat censored, somewhat biased) LLMs that we use today.

“Modeling knowledge” can mean modeling what people claim to be knowledge, taking explicit account of origins, evidence, controversies, provenance, social context, coherence, consilience, and consistency with other, well-supported facts. Compare the breadth and depth of the science of measuring the geoid to the information footprint of flat earth theories. Impartial representation bends toward truth.

Legal considerations

Regarding copyright and related legal concerns, knowledge models would gather and synthesize knowledge from multiple sources, with parallels to LLM pretraining, academic review articles, and search indexing. These transformative uses of information are generally acceptable under copyright law, and the representation of knowledge based on language-independent embeddings should be as well.

A vision and a goal

Advances in AI today center on tasks — increasing the ability to solve problems and provide services. Task-oriented AI amplifies capabilities that can transform the world, for both better and worse. Task AI, as presently conceived, does little to help us steer toward favorable outcomes.

Better knowledge can amplify task capabilities (accelerating the technologies of production, job displacement, medicine, and war), yet broader knowledge can also help us better understand the consequences of our actions. As AI capabilities expand, better knowledge integration becomes critical — not just for advancing technology, but for understanding and managing its potential consequences.

Knowledge-oriented AI can help us discover and explore options that are now unknown or known only in outline, and this kind of knowledge — by informing strategy — could help us steer away from catastrophe toward a future that is secure, open, and broadly appealing. The consequences of this kind of knowledge are uncertain, but the alternative — ignorance of our options — seems distinctly risky.

The development of knowledge-oriented AI calls for extending capabilities in new directions, toward broad, integrated knowledge and cumulative, collaborative reasoning. With a better knowledge of possibilities we may find our way to better outcomes.¹⁸

Whether or not actual knowledge models follow the approach I have outlined, I am confident that:

We can apply emerging AI capabilities to develop knowledge resources that are far more trustworthy, comprehensive, and accessible than any seen in human history.

• For researchers, this suggests developing architectures that integrate LLM capabilities with persistent, updatable latent-space knowledge stores.

• For system-builders, it means constructing knowledge bases that preserve provenance, maintain traceability, and enable ongoing refinement.

• For policy analysts, it underscores the urgent need for better knowledge infrastructure to surface coherent, viable proposals when incremental approaches fail. Shaping the exploration of policy space in shared LKMs can become a new, more agile, more powerful form of research and communication.

Some readers will find this vision compelling and actionable.

---

1. Hence the name of this Substack: “AI Prospects: Toward Global Goal Alignment”.

2. And, of course, catastrophic opportunities.

3. Note that what I mean by a “Large Knowledge Model” is related to yet quite different from another proposal of the same name, which focuses on “LLM-enhanced symbolic reasoning”.

4. Though inevitably, some will continue to enjoy wallowing in implausible conspiracy theories.

5. As explored in “The Platform: General Implementation Capacity”, implementation workflows span design, development, deployment, application, and adaptation, with each stage benefiting from integrated knowledge across multiple domains.

6. "Why Intelligence Isn't a Thing" discusses intelligence as a capacity, not a thing — a resource that can be focused, combined, and applied in different ways for different purposes. This has become increasingly obvious as AI advances from a speculative concept to a practical reality, yet tacit anthropomorphism remains strong.

7. There are, however, remarkably few informed experts. It’s a long story.

8. "AI Has Unblocked Progress Toward Generative Nanotechnologies” explains why AI-enabled protein engineering has removed long-standing obstacles on the most attractive path to atomically precise mass fabrication.

9. “Security Without Dystopia: Structured Transparency” outlines some of the principles and building blocks for such frameworks, but their potential remains underexplored due to siloed expertise across domains like cryptography, software security, institutional design, and sensor-based information systems.

10. Knowledge models could accelerate the development and adaptation of policy frameworks when mounting pressures demand change. I discuss prospects for delayed, accelerated change in “AI-Driven Strategic Transformation: Preparing to Pivot”.

11. “Paretotopian Goal Alignment” discusses how AI-enabled capabilities and prospects for greatly expanded material wealth could reshape incentives for cooperation among competing actors.

12. “Consilience” refers to “…the principle that evidence from independent, unrelated sources can ‘converge’ on strong conclusions.”

13. In Google’s words. See “Accelerating scientific breakthroughs with an AI co-scientist”.

14. Note that LLMs trained by companies, whether in the US, Europe, or China, give similar answers to most questions of fact. This isn’t surprising, because they have ingested — and digested — extensive, multiply-sourced information about a shared world. Reasoning while “filtering out semantic noise and seeking coherence guided by provenance, consensus, and especially consilience” should lead to improved knowledge representations that continue to align with one another, and with something that resembles reality.

15. Latent spaces encode semantic relationships as geometric relationships in high-dimensional vector spaces, allowing manipulation of concepts rather than text-strings. LLMs learn millions of language-independent, latent-space “concept vectors” in their internal reasoning, and activate dozens of these high-dimensional vectors per token-position. Typical vector dimensionalities are a thousand or more.

16. Recent results include compressed, semantically faithful, latent-space representations of text, sentence-level “Large Concept Models”, and latent-space chain-of-thought reasoning. All of these systems employ Transformer architectures with training grounded in token-prediction objectives.

17. Careful writers will know the struggle (and failure) to express any but the simplest or already-familiar concepts in a way that is accurate, precise, and concise — pick two. (This of course doesn’t precisely express the intended challenge and trade-off concept, but I hope you will understand what I’m trying to say.)

18. Please note that I am neither an optimist nor a pessimist regarding outcomes — I advocate exploring actionable possibilities, not judging odds like a spectator.